{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN-阿凯的机器学习玩具代码\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 鸢尾花数据集\n",
    "\n",
    "![u=2711639257,742490474&fm=27&gp=0.jpg](./IMG/u=2711639257,742490474&fm=27&gp=0.jpg)\n",
    "\n",
    "  鸢尾花数据是一个简易有趣的数据集。这个数据集来源于科学家在一岛上找到一种花的三种不同亚类别，分别叫做iris setosa, iris versicolor,iris virginica。但是这三个种类并不是很好分辩，所以他们又从花萼长度(sepal length)，花萼宽度(sepal width)，花瓣长度(petal length)，花瓣宽度(petal width)这四个角度测量不同的种类用于定量分析。基于这四个特征，这些数据成了一个多重变量分析的数据集。在sklearn中预置了鸢尾花（iris）的数据集， 我们可以直接导入， 每种各有50个样本，target（目标分类）数据分别为0,1,2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![iris_data_table.jpg](./IMG/iris_data_table.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入数据库\n",
    "from sklearn import datasets\n",
    "# 从数据库中载入iris样本\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "X = iris.data\n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.1,  3.5,  1.4,  0.2],\n",
       "       [ 4.9,  3. ,  1.4,  0.2],\n",
       "       [ 4.7,  3.2,  1.3,  0.2],\n",
       "       [ 4.6,  3.1,  1.5,  0.2],\n",
       "       [ 5. ,  3.6,  1.4,  0.2],\n",
       "       [ 5.4,  3.9,  1.7,  0.4],\n",
       "       [ 4.6,  3.4,  1.4,  0.3],\n",
       "       [ 5. ,  3.4,  1.5,  0.2],\n",
       "       [ 4.4,  2.9,  1.4,  0.2],\n",
       "       [ 4.9,  3.1,  1.5,  0.1]])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 打印前10组特征\n",
    "X[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 打印前10个数据的标签\n",
    "y[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了讲解KNN算法， 我们先从最简单的2D数据开始讲解。所以我们可以从iris数据集中选取两组特征值，例如花萼长度(sepal length)，花萼宽度(sepal width）。因为只选取了其中的两维数据， 信息肯定是丢失的，所以不必纠结与最后的准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.1,  3.5],\n",
       "       [ 4.9,  3. ],\n",
       "       [ 4.7,  3.2],\n",
       "       [ 4.6,  3.1],\n",
       "       [ 5. ,  3.6],\n",
       "       [ 5.4,  3.9],\n",
       "       [ 4.6,  3.4],\n",
       "       [ 5. ,  3.4],\n",
       "       [ 4.4,  2.9],\n",
       "       [ 4.9,  3.1]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = X[:,:2]\n",
    "data[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 打乱排序\n",
    "import numpy as np\n",
    "n_sample = len(data)\n",
    "# 初始化排序\n",
    "idxs = np.arange(n_sample)\n",
    "# 打乱排序\n",
    "np.random.shuffle(idxs)\n",
    "\n",
    "data = data[idxs]\n",
    "target = y[idxs]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2, 1, 2, 0, 2, 0, 2, 2, 0, 2, 1, 0, 2, 2, 2, 1, 1, 2, 0, 0, 2, 1,\n",
       "       0, 2, 1, 1, 2, 0, 0, 2, 1, 1, 0, 2, 1, 0, 0, 1, 2, 2, 0, 2, 2, 2, 1,\n",
       "       1, 1, 2, 2, 1, 1, 0, 0, 0, 2, 0, 0, 1, 2, 0, 0, 1, 2, 2, 1, 1, 0, 1,\n",
       "       0, 0, 0, 2, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 2, 2, 0, 1, 2, 1, 0, 1,\n",
       "       1, 1, 0, 2, 0, 0, 2, 2, 2, 0, 1, 2, 0, 2, 2, 0, 1, 0, 1, 2, 1, 2, 0,\n",
       "       0, 2, 1, 1, 1, 1, 0, 0, 2, 1, 2, 1, 2, 2, 0, 2, 1, 1, 1, 0, 2, 0, 2,\n",
       "       1, 0, 2, 0, 1, 1, 0, 1, 2, 1, 1, 2])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHkJJREFUeJzt3X9sHPd55/H3E1JrU6J/EDFR+SS5ak7nAkkaSxYl2WRT\nBNa1qRJDAS7BVQf4WssN3LRJk1wMBJeCSQOHDVrc+S6XCqhhOBacOG3sU9KeY1jnGlCAKtRZEmnJ\ncRIXhlK7VlSzZmzaskRaK9LP/TFLi1wvNbOc2dnvzH5ewILcmdnZZ78cPlx+95lnzN0REZFyeUe7\nAxARkewpuYuIlJCSu4hICSm5i4iUkJK7iEgJKbmLiJSQkruISAkpuYuIlFDi5G5mXWZ2zMweabDu\nVjObNLPjtdvHsw1TRESa0d3Etp8BngEuX2L9g+7+qaQ7u+qqq3z9+vVNPL2IiIyPj//C3fvjtkuU\n3M1sLfBh4M+Az6WMDYD169czNjaWxa5ERDqGmf1zku2STst8Dfg88OZFtvmomf3IzPaZ2bqE+xUR\nkRaITe5mdjPwkruPX2Sz7wPr3f19wOPA/Uvs63YzGzOzscnJyWUFLCIi8ZK8cx8CdprZ88B3gJvM\n7IGFG7j7y+5+rnb3XmBzox25+z3uPuDuA/39sVNGIiKyTLHJ3d2/4O5r3X09sAs44O63LNzGzK5e\ncHcn0QevIiLSJs1UyyxiZncCY+7+MPBpM9sJzAKvALdmE56IiCyHtetiHQMDA65qGRGR5pjZuLsP\nxG237HfuIpmZm4P9++HYMdi0CXbsgK6udkclUmhK7tJec3PwwQ/C4cNw9iysWgXbtsFjjynBi6Sg\n3jLSXvv3R4n9zBlwj74ePhwtF5FlU3KX9jp2LHrHvtDZs3D8eHviESkJJXdpr02boqmYhVatgo0b\n2xOPSEkouUt77dgRzbH39oJZ9HXbtmi5iCybPlCV9urqij483b8/morZuFHVMiIZUHKX9uvqgptv\njm4ikglNy4iIlJCSu4hICSm5i4iUkJK7iEgJKbmLiJSQkruISAkpuYuIlJCSu4hICekkJklP/dhF\ngqPkLumoH7tIkDQtI+moH7tIkJTcJR31YxcJkpK7pKN+7CJBUnKXdNSPXSRI+kBV0lE/dpEgKblL\neurHLhIcJfeyUw26SEdSci8z1aCLdCx9oFpmqkEX6VhK7mWmGnSRjqXkXmaqQRfpWEruZaYadJGO\npQ9Uy0w16CIdS8m97FSDLtKREk/LmFmXmR0zs0carLvEzB40sxNmdtjM1mcZpIiINKeZOffPAM8s\nse73gSl33wD8T+Av0gYmssjcHDzyCHzlK9HXubl2RyQStETTMma2Fvgw8GfA5xps8hHgy7Xv9wF7\nzMzc3bMIUjqcTsYSaVrSd+5fAz4PvLnE+jXASQB3nwVeA96ZOjoR0MlYIssQm9zN7GbgJXcfT/tk\nZna7mY2Z2djk5GTa3Umn0MlYIk1L8s59CNhpZs8D3wFuMrMH6rY5BawDMLNu4Arg5fodufs97j7g\n7gP9/f2pApcOopOxRJoWm9zd/Qvuvtbd1wO7gAPufkvdZg8Dv1f7/mO1bTTfLtnQyVgiTVt2nbuZ\n3QmMufvDwDeAb5nZCeAVoj8CItnQyVgiTbN2vcEeGBjwsbGxtjy3iEhRmdm4uw/EbafeMnJx1Sp8\n6UuwfXv0tVptd0QikoDaD8jSqlVYvRqmpqL7Bw7Anj0wMQGVSntjE5GL0jt3WdrIyIXEPm9qKlou\nIkFTcpeljY42Xn7oUL5xiEjTlNxlaUNDjZcPDuYbh4g0TcldljY8DH19i5f19UXLRSRoSu6ytEol\n+vD0i1+MqmW++EV9mCpSEKqWkYurVODOO9sdhYg0Scm9yGZmYPduOHIEtm6FvXuhp6fdUTVvbi46\n+/TYsaiPjM4+lYBkcXi24xBXci+qmRm4/HKYnY3uP/ccfPe7cPp0sRK8erVLwLI4PNt1iGvOvah2\n776Q2OfNzkbLi0S92iVgWRye7TrEldyL6siRxsuPHs03jrTUq10ClsXh2a5DXMm9qLZubbx8y5Z8\n40hLvdolYFkcnu06xJXci2rvXuiu+8ikuztaXiTq1S4By+LwbNchrpa/RTZfLXP0aPSOvejVMurV\nLgHK4vDM8hBP2vJXyV1EpECSJneVQhZZHgW4qkEXKSQl96LKowBXNegihaUPVIsqjwJc1aCLFJaS\ne1HlUYCrGnSRwlJyL6o8CnBVgy5SWEruRZVHAa5q0EUKS6WQRZZHAa5q0EWCojp3EZESUp17GqHU\ndocSh0gL6PBuLSX3eqHUdocSh0gL6PBuPX2gWi+U2u5Q4hBpAR3erafkXi+U2u5Q4hBpAR3erafk\nXi+U2u5Q4hBpAR3erafkXi+U2u5Q4hBpAR3eradSyEZCqe0OJQ6RFtDhvTyZ1bmb2aXAPwCXEFXX\n7HP3P63b5lbgvwGnaov2uPu9F9tv0MldRCRQWda5nwNucvczZrYC+KGZ7Xf3J+q2e9DdP7WcYGUJ\n1SqMjMDoKAwNwfAwVCrJ10M+xcQqWBYJTmxy9+it/Zna3RW1W3vmcjpJtQqrV8PUVHT/wAHYswcm\nJqIEHrce8ikmVsGySJASfaBqZl1mdhx4CXjc3Q832OyjZvYjM9tnZusyjbITjYxcSNzzpqai5UnW\nQz7FxCpYFglSouTu7nPuvhFYC2w1s/fWbfJ9YL27vw94HLi/0X7M7HYzGzOzscnJyTRxl9/oaOPl\nhw4lWw/5FBOrYFkkSE2VQrr7q8APgN+uW/6yu5+r3b0X2LzE4+9x9wF3H+jv719OvJ1jaKjx8sHB\nZOshn2JiFSyLBCk2uZtZv5ldWfu+B/hN4B/rtrl6wd2dwDNZBtmRhoehr2/xsr6+aHmS9ZBPMbEK\nlkWClKRa5mrgfjPrIvpj8JC7P2JmdwJj7v4w8Gkz2wnMAq8At7Yq4I5RqUQfjo6MRFMtg4OLq2Hi\n1kP0geZjj7W2mDiP5xCRpukkJhGRAlE/9zSyqNtOUoOedh9J4kz7WrJ4HYGYe3OO/Sf2c+zFY2y6\nehM7Nuyg6x3JxyKP4RbJjLu35bZ582YP0uys+/bt7r297mbR1+3bo+VJnTvn3tfnHhUHRre+vmh5\nVvtIEmfa15LF6wjE7Nysb79/u/d+tdfty+a9X+317fdv99m5ZGORx3CLJEE0HR6bY9U4rF4WddtJ\natDT7iNJnGlfSxavIxD7T+zn8KnDnKmewXHOVM9w+NRh9p9INhZ5DLdIlpTc62VRt52kBj3tPpLE\nmfa1ZPE6AnHsxWOcrS4ei7PVsxyfSDYWeQy3SJaU3OtlUbedpAY97T6SxJn2tWTxOgKx6epNrKos\nHotVlVVsXJ1sLPIYbpFMJZm7acVNc+4p96E596Zozl3KgoRz7iqFbCSLRtPzVSZL1aBnsY8kcaZ9\nLVm8jkDMV8scnzjOxtUbl10t08rhFomTWT/3Vgk6uYuIBEp17mUQVzStourgVM/PMXLfOKNHphna\nupLh2zZTWZHvz6REpyZICnrnHqq4Punqox6c6vk5Vm98iqkT10K1Byoz9G14lonj1+WW4Ovb/EPU\ncmhhm38ptqTv3FUtE6q4omkVVQdn5L7xWmLvBbqg2svUiWsZuW88vxjKc2qCpKTkHqq4omkVVQdn\n9Mh09I59oWoPh45O5xdDeU5NkJSU3EMVVzStourgDG1dCZWZxQsrMwxuWZlfDOU5NUFS0px7qDTn\nXjiac5c8qBSyDOKKplVUHZz5aplDR6cZ3NLeapkSnJogDSi5i4iUUOfWuWdR+51kH3kUE6uOPbG0\nvdrzkqQOPo8fe9pTKLIYbx3eLZakR0Erbi3pLZNFc48k+8ij54oalSSWtm9MXs5VZ73v3eNO5XWH\nWafyuve9e9zPVS/EmcePPe45YtdnMN46vJePjuznnkXtd5J95FFMrDr2xNL2as9Lkjr4PH7saU+h\nyGK8dXi3XrmSexa130n2kUcxserYE0vbqz0vSerg8/ixpz2FIovx1uHdeuVK7lnUfifZRx7FxKpj\nTyxtr/a8JKmDz+PHnvYUiizGW4d3DpLM3bTipjn3HF5Lh9Cce3M0515sdGw/9yxqv5PsI49iYtWx\nJ5a2V3tektTB5/FjT3sKRRbjrcN7eVTnLiJSQuoKKSLSwZTcG5mbg0cega98Jfo6N5f9PmZmYNcu\neNe7oq8zM433I6WSxaEVZ+ZclV1ffYB3/Ye97PrqA8ycqza/jzfm2PXJZ3nX5ufY9clnmXmjBYFK\nS2lapl4WDbni9jEzA5dfDrOzFx7T3Q2nT0NPz9L7lULLo9fbzLkql7/n/zF78nqoroTKNN3rnuT0\nT26k55JknwnNvDHH5e88y+z0ZW8t6175OqdfXkXPpZoUbzdNyyxXHidC7d69OLFDdH/37uxehwQn\njxN3dt/1UC2xX0Z0otRlzJ68nt13PZR8H3f8rJbY7a3b7PRl7L7jZ9kFKi2n5F4vjxOhjhxp/Lij\nR5M/hxROHifuHBk7H71jX6i6kqPj55Pv44kVDZcfXWK5hEnJvV4eJ0Jt3dr4cVu2JH8OKZw8TtzZ\nOrACKnVXfqpMs2Vz8sS89YbGfwi2LLFcwqQ593qac5cW0Zy7ZEF17mnkcSLUzEw0x370aPSOfe9e\nJfYOkMeJOzPnquy+6yGOjp9ny+YV7L3jPyZO7G/t4405dt/xM44+sYItN5xn713/Vok9EJkldzO7\nFPgH4BKi/u/73P1P67a5BPgmsBl4Gfgdd3/+YvsNOrmLiAQqy4t1nANucvczZrYC+KGZ7Xf3JxZs\n8/vAlLtvMLNdwF8Av7OsyOOkvcpAXuIu5pEkzgBeSxYXZajOVhk5OMLoC6MMXTPE8PuHqXRfGItM\nniODi2Ak2Uce5v+pO3Ik+nim/p+6uPEqywVB8lKUOJuWpAHN/A1YCTwJbKtb/hhwY+37buAX1P4r\nWOq2rMZhaTse5SWusViSOAN4LVk0iDp3/pz3/Xmf82XeuvX9eZ+fO38uu+fIoCFXkn3kYXravbt7\n8aHT3R0td48fr7I0J8tLUeJciCwv1mFmXWZ2HHgJeNzdD9dtsgY4WftjMQu8Brwz7R+et0l7lYG8\nxF3MI0mcAbyWLC7KMHJwhKk3Fo/F1BtTjBwcye45MrgIRpJ95CHuFIi48SrLBUHyUpQ4lyNRcnf3\nOXffCKwFtprZe5fzZGZ2u5mNmdnY5ORk8ztIe5WBvMRdzCNJnAG8liwuyjD6QuOxOHTyUHbPkcFF\nMJLsIw9xp0DEjVdZLgiSl6LEuRxN1bm7+6vAD4Dfrlt1ClgHYGbdwBVEH6zWP/4edx9w94H+/v7m\no017lYG8xF3MI0mcAbyWLC7KMHRN47EYXDeY3XNkcBGMJPvIQ9wpEHHjVZYLguSlKHEuS9y8DdAP\nXFn7vgc4CNxct80ngbtr3+8CHorbr+bcNeee2XNozl1z7iWPcyGyuliHmb0PuB/oInqn/5C732lm\nd9ae5OFaueS3gE3AK8Aud/+ni+132aWQaa8ykJe4i3kkiTOA15LFRRnmq2UOnTzE4LrBJatlUj1H\nBhfBSLKPPMSdAhE3XmW5IEheihLnPJ3EJCJSQlnWuRdLYYtSyyuuLjuLOvc84kgSZ2zNdAavNa/x\nCoF+nZevXMk9j+Yd0pS5N+f44AMf5PCpw5ytnmVVZRXb1mzjsVseo+sdXbHrQ4kjSZxxh18WrzWv\n8QqBfp3TKVdXyCIXpZZUXF12FnXuecSRJM7YmukMXmte4xUC/TqnU67kXuSi1JKKq8vOos49jziS\nxBlbM53Ba81rvEKgX+d0ypXcC12UWk5xddlZ1LnnEUeSOGNrpjN4rXmNVwj065xOuZL7jh3RpFxv\nL5hFX7dti5ZLW+zYsINta7bRW+nFMHorvWxbs40dG3YkWh9KHEnijDv8sniteY1XCPTrnE75SiFD\nK0qV2LrsLOrc84gjSZyxNdMZvNa8xisE+nV+O9W5i4iUUOfWuUtwYvuox/R7T7KPLMT2nc+gBX9c\nm3/Ip469LLXyIfSlD5XeuUtLxdUqV2errL5r9aK2wH2X9jFxx8RbiTWPeue4OJLEEPtaq7B69eJu\n0H19MDFxIcHnUcdellr5PI6LEGvtk75zL9cHqhKc2D7qMf3ek+wjC7F95zNowR/X5h/yqWMvS618\nCH3pQ6bkLi0V20c9pt97kn1kIbbvfAYt+OPa/EM+dexlqZUPoS99yJTcpaVi+6jH9HtPso8sxPad\nz6AFf1ybf8injr0stfIh9KUPmebcpaU0564591bRnHvMdkru0mqxfdRj+r0n2UcWYvvOZ9CCP67N\nP+RTx16WWvkQ+tLnTcldRKSEVOcuQBj1zFnEcGa6yoc+McrTY1fwawOv8ejdQ/SurMQ/MOM4Yp8j\nrp97QWumpXiU3EsshLnVLGI4M13lsqvOwswHADj4DFy271Ve/wWJE3wuc9lx/dwDnL+V8lK1TImF\nUM+cRQwf+sQozFwJ2IXbzJXR8hzjiH2OuH7uBa6ZluJRci+xEOqZs4jh6bErGi7/8Xjj5a2KI/Y5\n4vq5F7hmWopHyb3EQqhnziKGXxt4reHy925uvLxVccQ+R1w/9wLXTEvxKLmXWAi9v7OI4dG7h6Dn\nVcAv3HpejZbnGEfsc8T1c1d/csmRSiFLLoR65iximK+W+fH4Fbx3c7pqmZbWj8f1cw+sZlqKR3Xu\nIiIlpDp3yU3a+vEkjw+hRl06U1GPCyV3SSVt/XiSx4dQoy6dqcjHhT5QlVTS1o8neXwINerSmYp8\nXCi5Sypp68eTPD6EGnXpTEU+LpTcJZW09eNJHh9Cjbp0piIfF0rukkra+vEkjw+hRl06U5GPC5VC\nSmpp68eTPD6EGnXpTKEdF6pzFxEpoczq3M1sHfBN4JeIzv2+x93/V902HwD+D/BcbdH33P3OZoPu\nJFnUzobQqz1JHLHrC1pH3Ej1/Bwj940zemSaoa0rGb5tM5UVOZ8RXKLxlOVLUuc+C9zh7k+a2WXA\nuJk97u4/rdvuoLvfnH2I5ZNF7WwIvdqTxBG7vsB1xPWq5+dYvfEppk68G6o9HHhghj1ff4qJ49fl\nluDLNJ6STuwHqu7+ors/Wfv+deAZYE2rAyuzLGpnQ+jVniSO2PUFriOuN3LfOFMnroVqL9AF1V6m\nTlzLyH3jucVQpvGUdJqqljGz9cAm4HCD1Tea2VNmtt/M3rPE4283szEzG5ucnGw62LLIonY2hF7t\nSeKIXV/gOuJ6o0emodqzeGG1h0NHp3OLoUzjKekkTu5m1gt8F/isu5+uW/0k8Mvufh3wl8DfNdqH\nu9/j7gPuPtDf37/cmAsvi9rZEHq1J4kjdn2B64jrDW1dCZWZxQsrMwxuWZlbDGUaT0knUXI3sxVE\nif3b7v69+vXuftrdz9S+fxRYYWZXZRppiWRROxtCr/YkccSuL3Adcb3h2zbTt+FZqJwB5qByhr4N\nzzJ82+bcYijTeEo6saWQZmbA/cAr7v7ZJbZZDfyru7uZbQX2Eb2TX3LnnV4KmUXtbAi92pPEEbs+\nsDriNOarZQ4dnWZwS3urZcownvJ2mdW5m9mvAweBp4E3a4v/BLgGwN3vNrNPAX9IVFkzA3zO3Q9d\nbL+dntxFRJYjszp3d/8h0SXnL7bNHmBP8vAklBr1LFRnq4wcHGH0hVGGrhli+P3DVLqbu0qSiGRL\n/dzbIJQa9SxUZ6usvms1U29MAXDg+QPsObKHiTsmlOBF2kiNw9oglBr1LIwcHHkrsc+bemOKkYMj\nbYpIREDJvS1CqVHPwugLow2XHzp50Y9cRKTFlNzbIJQa9SwMXTPUcPngusGcIxGRhZTc2yCUGvUs\nDL9/mL5L+xYt67u0j+H3D7cpIhEBfaDaFl3v6OKxWx4LokY9rUp3hYk7Jhg5OMKhk4cYXDeoahmR\nAKifu4hIgWRW515KBWl4XZRa+KLEmQeNhYSi85J7QRpeF6UWvihx5kFjISHpvA9UC9Lwuii18EWJ\nMw8aCwlJ5yX3gjS8LkotfFHizIPGQkLSecm9IA2vi1ILX5Q486CxkJB0XnIvSMProtTCFyXOPGgs\nJCSdWQpZkIbXofRrj1OUOPOgsZBWy6yfe6uozl1EpHlJk3vnTcuIiHSAzqtzl441fwm80SPTDG1t\n/hJ4OkFJikTJXTpC9fwcqzc+xdSJd0O1hwMPzLDn608xcfy6RAleJyhJ0WhaRjrCyH3jTJ24Fqq9\nQBdUe5k6cS0j940nerxOUJKiUXKXjjB6ZBqqPYsXVns4dHQ60eN1gpIUjZK7dIShrSuhMrN4YWWG\nwS0rEz1eJyhJ0Si5S0cYvm0zfRuehcoZYA4qZ+jb8CzDt21O9HidoCRFow9UpSNUVnQxcfw6Ru4b\n59DRaQa3NFctU6YLrEhn0ElMIiIFopOYREQ6mJK7iEgJKbmLiJSQkruISAkpuYuIlJCSu4hICSm5\ni4iUUGxyN7N1ZvYDM/upmf3EzD7TYBszs6+b2Qkz+5GZXd+acEVEJIkkZ6jOAne4+5NmdhkwbmaP\nu/tPF2yzA/h3tds24K9qXyUF9Q8XkeWKTe7u/iLwYu37183sGWANsDC5fwT4pkenuz5hZlea2dW1\nx8oyqH+4iKTR1Jy7ma0HNgGH61atAU4uuP/z2jJZJvUPF5E0Eid3M+sFvgt81t1PL+fJzOx2Mxsz\ns7HJycnl7KJjqH+4iKSRKLmb2QqixP5td/9eg01OAesW3F9bW7aIu9/j7gPuPtDf37+ceDuG+oeL\nSBpJqmUM+AbwjLv/jyU2exj43VrVzA3Aa5pvT0f9w0UkjSTVMkPAfwaeNrP5OYE/Aa4BcPe7gUeB\nDwEngGlgd/ahdhb1DxeRNNTPXUSkQNTPXUSkgym5i4iUkJK7iEgJKbmLiJSQkruISAkpuYuIlJCS\nu4hICbWtzt3MJoF/bsuTX3AV8Is2x5CE4syW4sxOEWKEcsX5y+4e27+lbck9BGY2luRkgHZTnNlS\nnNkpQozQmXFqWkZEpISU3EVESqjTk/s97Q4gIcWZLcWZnSLECB0YZ0fPuYuIlFWnv3MXESmljkju\nZtZlZsfM7JEG6241s0kzO167fbwdMdZied7Mnq7F8bZ+yLWLoXzdzE6Y2Y/M7PoAY/yAmb22YDy/\nlHeMtTiuNLN9ZvaPZvaMmd1Yt77tY5kwzraPp5n96oLnP25mp83ss3XbtH08E8bZ9vGsxfFfzOwn\nZvZjM/sbM7u0bv0lZvZgbTwP165f3Rx3L/0N+Bzw18AjDdbdCuxpd4y1WJ4HrrrI+g8B+wEDbgAO\nBxjjBxqNcxvivB/4eO37CnBlaGOZMM4gxnNBPF3ABFGtdXDjmSDOto8nsAZ4Duip3X8IuLVumz8C\n7q59vwt4sNnnKf07dzNbC3wYuLfdsWTgI8A3PfIEcKWZXd3uoEJjZlcAv0F0eUjcverur9Zt1vax\nTBhnaLYDP3P3+hMQ2z6edZaKMxTdQI+ZdQMrgX+pW/8Roj/8APuA7bVLniZW+uQOfA34PPDmRbb5\naO1fyX1mtu4i27WaA39vZuNmdnuD9WuAkwvu/7y2LE9xMQLcaGZPmdl+M3tPnsHV/AowCeytTcfd\na2ar6rYJYSyTxAntH8+FdgF/02B5COO50FJxQpvH091PAf8deAF4keia039ft9lb4+nus8BrwDub\neZ5SJ3czuxl4yd3HL7LZ94H17v4+4HEu/LVsh1939+uBHcAnzew32hjLUuJifJLoX+HrgL8E/i7v\nAIneFV0P/JW7bwLOAv+1DXHESRJnCOMJgJlVgJ3A/25XDEnExNn28TSzPqJ35r8C/BtglZndkvXz\nlDq5E13ce6eZPQ98B7jJzB5YuIG7v+zu52p37wU25xviolhO1b6+BPwtsLVuk1PAwv8s1taW5SYu\nRnc/7e5nat8/Cqwws6vyjJHoXePP3f1w7f4+oiS6UNvHkgRxBjKe83YAT7r7vzZYF8J4zlsyzkDG\n898Dz7n7pLufB74HDNZt89Z41qZurgBebuZJSp3c3f0L7r7W3dcT/Zt2wN0X/YWsmxfcCTyTY4gL\n41hlZpfNfw/8FvDjus0eBn63VplwA9G/cy+GFKOZrZ6fGzSzrUTHWFMHZVruPgGcNLNfrS3aDvy0\nbrO2jmXSOEMYzwX+E0tPdbR9PBdYMs5AxvMF4AYzW1mLZTtvzzsPA79X+/5jRLmrqZOSulOHWUBm\ndicw5u4PA582s53ALPAKUfVMO/wS8Le1464b+Gt3/79m9gkAd78beJSoKuEEMA3sDjDGjwF/aGaz\nwAywq9mDMiN/DHy79i/6PwG7AxvLpHEGMZ61P+a/CfzBgmXBjWeCONs+nu5+2Mz2EU0RzQLHgHvq\n8tI3gG+Z2QmivLSr2efRGaoiIiVU6mkZEZFOpeQuIlJCSu4iIiWk5C4iUkJK7iIiJaTkLiJSQkru\nIiIlpOQuIlJC/x9sNtldGRwCLwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5f21580860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "\n",
    "def drawData(data, target):\n",
    "    targetColorDict = {\n",
    "        0: 'red',\n",
    "        1: 'green',\n",
    "        2: 'blue'\n",
    "    }\n",
    "\n",
    "    for t,color in targetColorDict.items():\n",
    "        idxs = np.argwhere(target == t)\n",
    "        idxs.resize(len(idxs))\n",
    "        plt.scatter(data[idxs][:,0],data[idxs][:,1],s=25,marker='o', c=targetColorDict[t])\n",
    "\n",
    "    plt.show()\n",
    "    \n",
    "drawData(data, target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据归一化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果维度之间的数值的差别很大的话， 不能直接进行距离计算，例如这样的\n",
    "\n",
    "$$distance = \\sqrt{(0-67)^2 + (20000 - 32000)^2 + (1.1 - 0.1)^2}$$\n",
    "\n",
    "需要对其进归一化（normalized）， 将数值范围放缩到0-1之间。\n",
    "$$newValue = (oldValue - minValue) / (maxValue - minValue)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHWpJREFUeJzt3X+MH/Wd3/Hnm12+wfYasoo5OQL7DLEtnQuNje112FV6\ntM4Vm56w1ESROXG9GBKUazhVCq0EikMR3ktzPdEfKegoau0khy7El5Miy7VrqQcUtA72emuT8EO4\nCwnYHC4Ot0DsXfvrXd79Y76Gr5ddz2d3fnxn5vt6SF/tfuc73/m+Z76zb48/8573mLsjIiLVckmr\nAxARkfQpuYuIVJCSu4hIBSm5i4hUkJK7iEgFKbmLiFSQkruISAUpuYuIVJCSu4hIBXW26oMXLFjg\nS5YsadXHi4iU0tDQ0K/d/cq4+VqW3JcsWcKhQ4da9fEiIqVkZq+HzKdhGRGRClJyFxGpICV3EZEK\nUnIXEakgJXcRkQpSchcRqSAldxGRCopN7ma23czeNrMXpnndzOx7ZjZsZj83sxvSD1Mqb2ICdu+G\nbduinxMTrY5IpNRCLmL6PvAw8MNpXt8ILGs81gF/0fgpEmZiAm6+GQ4cgNOnYd48WLcO9u2Djo5W\nRydSSrFH7u7+DPD3F5llE/BDjzwHfNLMPp1WgNIG9u6NEvupU+Ae/TxwIJouIrOSxpj7VcCxpufH\nG9M+xszuMrNDZnbo5MmTKXy0VMLhw9ERe7PTp+HIkdbEI1IBuZ5QdffH3H2Nu6+58srYvjfSLlat\nioZims2bBytXtiYekQpII7m/CSxqen51Y5pImI0bozH2ri4wi36uWxdNF5FZSaMr5C7gbjN7guhE\n6nvu/lYKy5V20dERnTzduzcailm5MkrsOpkqMmuxyd3MfgTcBCwws+PAvwUuBXD3R4E9wC3AMDAK\nbMkqWKmwjg74/d+PHiKSWGxyd/fbYl534BupRSQiIonpClURkQpSchcRqSAldxGRClJyFxGpICV3\nEZEKUnIXEakgJXcRkQpSchcRqaA02g+IRD3Z9+6NOjyuWqX2ASItpuQuyelmGyKFo2EZSU432xAp\nHCV3SU432xApHCV3SU432xApHCV3SU432xApHJ1QleR0sw2RwlFyl3ToZhsihaLk3g5Ugy7SdpTc\nq0416CJtSSdUq0416CJtScm96lSDLtKWlNyrTjXoIm1Jyb3qVIMu0pZ0QrXqVIMu0paU3NuBatBF\n2o6Su5SH6vVFgim5SzmoXl9kRnRCVcpB9foiM6LkLuWgen2RGVFyl3JQvb7IjAQldzPbYGavmNmw\nmd07xeuLzewpMztsZj83s1vSD1Xamur1RWYk9oSqmXUAjwC/BxwHBs1sl7u/1DTbVmCnu/+Fma0A\n9gBLMohX2pXq9UVmJKRapgcYdvfXAMzsCWAT0JzcHbi88fsVwN+lGaQIoHp9kRkIGZa5CjjW9Px4\nY1qzB4Dbzew40VH7n6QSnRRDvQ733w/r10c/6/VWRyQiMdKqc78N+L67P2RmNwJ/aWbXufsHzTOZ\n2V3AXQCLFy9O6aMlU/U6LFwIIyPR8yefhIcfhhMnoFZrbWwiMq2QI/c3gUVNz69uTGt2J7ATwN1/\nBlwGLJi8IHd/zN3XuPuaK6+8cnYRS776+z9K7OeNjETTRaSwQpL7ILDMzK4xsxqwGdg1aZ43gPUA\nZvY7RMn9ZJqBSosMDEw9ff/+fOMQkRmJTe7uPg7cDewDXiaqinnRzB40s1sbs90DfM3Mngd+BHzF\n3T2roCVHfX1TT+/tzTcOEZkRa1UOXrNmjR86dKglny0zMHnMHaC7W2PuIi1iZkPuviZuPl2hKhdX\nq0WJ/Nvfjqplvv1tJXaRElBXSIlXq8GDD7Y6ChGZASX3shsbgy1b4OBB6OmBHTtgzpxWRzVz6tUu\nBZXGrtmK3VvJvczGxuDyy2F8PHr+y1/C3/wNvP9+uRK8erVLQaWxa7Zq99aYe5lt2fJRYj9vfDya\nXibq1S4Flcau2ardW8m9zA4enHr64GC+cSSlXu1SUGnsmq3avZXcy6ynZ+rpa9fmG0dS6tUuBZXG\nrtmq3VvJvcx27IDOSadNOjuj6WWiXu1SUGnsmq3avXURU9mdr5YZHIyO2MteLaNe7VIwaeyaae7e\noRcxKbmLiJRIaHJXKWTZ5VGEqxp0kdJRci+zPIpwVYMuUko6oVpmeRThqgZdpJSU3MssjyJc1aCL\nlJKSe5nlUYSrGnSRUlJyL7M8inBVgy5SSiqFLLs8inBVgy5SGKpzFxGpINW5p6EI9d1FiEEkA9q1\ns6XkPp0i1HcXIQaRDGjXzp5OqE6nCPXdRYhBJAPatbOn5D6dItR3FyEGkQxo186ekvt0ilDfXYQY\nRDKgXTt7Su7TKUJ9dxFiEMmAdu3sqRTyYopQ312EGEQyoF17dlTnLiJSQaHJXcMyRVevw/33w/r1\n0c96fWavQ3SItHs3bNsW/ZyYSD/OPD5DRILpyL3I6nVYuBBGRj6a1t0NJ05ArRb/OuRTUKyiZZHc\n6Mi9Cvr7L0zcED3v7w97HfIpKFbRskjhKLkX2cDA1NP37w97HfIpKFbRskjhBCV3M9tgZq+Y2bCZ\n3TvNPF82s5fM7EUz+6t0w2xTfX1TT+/tDXsd8ikoVtGySOHEjrmbWQdwFPg94DgwCNzm7i81zbMM\n2An8E3cfMbPfcve3L7ZcjbkH0Ji7iEySZlfIHmDY3V9rLPgJYBPwUtM8XwMecfcRgLjELoFqtShR\n9/dHQy29vbB160eJO+51iJLrvn3ZFhTn8RkiMiMhR+5fAja4+1cbz/8QWOfudzfN81Oio/s+oAN4\nwN3/58WWqyN3EZGZy7ufeyewDLgJuBp4xsyud/d3JwV1F3AXwOLFi1P66AwlbThdr0dH1QMD0fj4\n5KPqNJYREmMR1qMAJj6YYO/wXg6/dZhVn17FxqUb6bhkZv+7iNuU6lEuheHuF30ANwL7mp7fB9w3\naZ5HgS1Nz/8WWHux5a5evdoLbXzcff16964ud7Po5/r10fQQZ8+6d3e7R8WB0aO7O5oeKm4ZITEW\nYT0KYHxi3Nf/YL13fafL7QHzru90+fofrPfxicDt4PGbMummFgkBHPKYvO3uQdUyg8AyM7vGzGrA\nZmDXpHl+SnTUjpktAJYDryX7Z6fFktZuh9SgJ11GSIxFWI8C2Du8lwNvHuBU/RSOc6p+igNvHmDv\ncHgtftymVLm/FElscnf3ceBuYB/wMrDT3V80swfN7NbGbPuAd8zsJeAp4N+4+ztZBZ2LpLXbITXo\nSZcREmMR1qMADr91mNP1C7fD6fppjpwIr8WP25Qq95ciCapzd/c97r7c3T/j7n/amHa/u+9q/O7u\n/k13X+Hu17v7E1kGnYuktdshNehJlxESYxHWowBWfXoV82oXbod5tXmsXBheix+3KVXuL4USMnaT\nxUNj7iksQ2PuwTTmLlVB4Ji7GoddTNKG0+erTKarQU9jGSExFmE9CuB8tcyRE0dYuXBlomqZ6Tal\nepRL1tTPXUSkgvKuc5esqLC6VOrnJujfPsTAwVH6euay9Y7V1C7N//uoyKUJkoCO3IssrmeLeroU\nSv3cBAtXPs/I8HKoz4HaGN1Lj3LiyGdzTfAhLYekvNTPvQpUWF0q/duHGom9C+iAehcjw8vp3z6U\nbxzVuDRBElJyLzIVVpfKwMHR6Ii9WX0O+wdH842jGpcmSEJK7kWmwupS6euZC7WxCyfWxuhdOzff\nOKpxaYIkpDH3ItOYe6lozF3yoFLIqlBhdamcr5bZPzhK79rWV8uU/NIEmYKSu4hIBanOPY3677hl\n5FFMrDr2YGn0a89DXC18Hl95GpdPJN3e2rUzFtKjIItHpr1l0mjyEbeMPHquqFlJsDR6x+ThbH3c\nu1cMObXfOIw7td9494ohP1uP4szjK0+jR07S7a1de/ZIsZ97+aRR/x23jDyKiVXHHiyNfu15iKuF\nz+MrT+PyiaTbW7t29qqZ3NOo/45bRh7FxKpjD5ZGv/Y8xNXC5/GVp3H5RNLtrV07e9VM7mnUf8ct\nI49iYtWxB0ujX3se4mrh8/jK07h8Iun21q6dg5CxmyweGnPPaT3ahMbcw2nMvdxo+37uadR/xy0j\nj2Ji1bEHS6Nfex7iauHz+MrTuHwi6fbWrj07qnMXEakgdYVMw8QE7N4N27ZFPycm0n//2Bhs3gzX\nXhv9HBv7+DxSKUl3qxBjZ+ts/s7jXPvPd7D5O48zdrY+s/efmWDzN45y7epfsvkbRxk7k0GQkikd\nuU8nad+WkPePjcHll8P4+Efv6+yE99+HOXOmXq6UWh7tgMbO1rn8H/yM8WM3QH0u1EbpXPR/eP/F\nG5nzifhhw7EzE1z+qdOMj87/cFrn3N/w/jvzmHOZxk1aTUfuSSUtxA15/5YtFyZ2iJ5v2ZLeekih\n5FHfveWhnY3EPp+oln4+48duYMtDO8Pef8+rjcRuHz7GR+ez5Z5X0wtSMqfkPp2khbgh7z94cOr3\nDg6Gxymlkkd998FD56Ij9mb1uQwOnQt7/3OXTjl9cJrpUkxK7tNJWogb8v6enqnfu3ZteJxSKnnU\nd/esuRRqk24QUhtl7eqw5Nzzuan/EVg7zXQpJo25T0dj7pIBjblLUiqFTEPSQtyQ94+NRWPsg4PR\nEfuOHUrsFZdHfffY2TpbHtrJ4NA51q6+lB33fDkosX/4/jMTbLnnVQafu5S1nzvHjoc+o8ReEEru\nIiIVVP1+7mk0pM5aSL/3EqxHGn3S6+N1+p/tZ+CNAfoW97H181updX60LVL5jBT6pMctIw/n/zN3\n8GB0Wmbyf+ZCtlVVesbnoSxxzlhIj4IsHol6y6TRHCNrIb1nSrAeafRsOXvurHd/t9t5gA8f3d/t\n9rPnzqb3GSn0bIlbRh5GR907Oy/cbTo7o+nuYduqKv1r8lCWOJtR6X7uaTSkzlpIv/cSrEcafdL7\nn+1n5MyF22LkzAj9z/an9xkp9EmPW0Ye4i59CNlWVekZn4eyxDkbQcndzDaY2StmNmxm915kvi+a\nmZtZ7HhQImk0pM5aSL/3EqxHGn3SB96YelvsP7Y/vc9IoU963DLyEHfpQ8i2qkrP+DyUJc7ZiE3u\nZtYBPAJsBFYAt5nZiinmmw/8K+BA2kF+TBoNqbMW0u+9BOuRRp/0vsVTb4veRb3pfUYKfdLjlpGH\nuEsfQrZVVXrG56Escc5K3LgNcCOwr+n5fcB9U8z3n4B/BjwNrIlbrsbcy7EeGnPXmPtslGDXLlWc\nzUirn7uZfQnY4O5fbTz/Q2Cdu9/dNM8NwLfc/Ytm9jTwr939Y3WOZnYXcBfA4sWLV7/++uuz/1cp\njYbUWQvp916C9UijT/r5apn9x/bTu6h32mqZRJ+RQp/0uGXkIe7Sh5BtVZWe8XkoS5znpVbnHpfc\nzewS4EngK+7+q4sl92aqcxcRmbk069zfBBY1Pb+6Me28+cB1wNNmBrAQ2GVmt8Yl+EyVtji1muJq\ns9Ooc08aQxpxhux2Sdc1j21VFPoznr2Q5D4ILDOza4iS+mbgD86/6O7vAQvOPw89cs9UHg08JNjE\nBxPc/PjNHHjzAKfrp5lXm8e6q9ax7/Z9dFzSEft6HjGkEWfIbpd0XfPYVkWhP+NkYqtl3H0cuBvY\nB7wM7HT3F83sQTO7NesAZ6XMxakVFFebnUade9IY0ogzZLdLuq55bKui0J9xMkF17u6+x92Xu/tn\n3P1PG9Pud/ddU8x7U0uP2qHcxakVFFebnUade9IY0ogzZLdLuq55bKui0J9xMuW8QjVOqYtTqyeu\nNjuNOvekMaQRZ8hul3Rd89hWRaE/42Sqmdw3bowG57q6wCz6uW5dNF1yt3HpRtZdtY6uWheG0VXr\nYt1V69i4dGPQ63nEkEacIbtd0nXNY1sVhf6Mk6luy9+iFae2ubja7DTq3JPGkEacIbtd0nXNY1sV\nhf6MP0793EVEKqj6/dylVOLqleP6vYcsIw2xfedT6P0d1+a/KDX/ZVCEvvRFpSN3yVxcvXJ9vM7C\nhxZe0Ba4+7JuTtxz4sPEmkfNc1wccTGExFivw8KFF3aD7u6GEyeiBF+Umv8yyGOfKGKtfeiRezVP\nqEqhxNUrx/V7D1lGGmL7zqfQ+zuuzX9Rav7LoAh96YtMyV0yF1evHNfvPWQZaYjtO59C7++4Nv9F\nqfkvgyL0pS8yJXfJXFy9cly/95BlpCG273wKvb/j2vwXpea/DIrQl77INOYumdOYu8bcs6Ax95j5\nlNwlD3H1ynH93kOWkYbYvvMp9P6Oa/NflJr/MihCX/q8KbmLiFSQ6tzlQ0WoaU4jhlOjdW75+gC/\nOHQF1695jz2P9tE1txb/xhRjiP2MkH7uJa2blnJRcq+4IoyvphHDqdE68xechrGbAHj2ZZj/k3f5\nza8JSvC5jGWH9HMv4BiuVJOqZSquCDXNacRwy9cHYOyTgH30GPtkND2nGGI/I6Sfe4nrpqVclNwr\nrgg1zWnE8ItDV0w5/YWhqadnEUPsZ4T0cy9x3bSUi5J7xRWhpjmNGK5f896U069bPfX0LGKI/YyQ\nfu4lrpuWclFyr7gi9P9OI4Y9j/bBnHcB/+gx591oek4xxH5GSD939SiXnKgUsg0UoaY5jRjOV8u8\nMHQF162efbVMpvXjIf3cC1Y3LeWiOncRkQpSnbvkKmkNecj7i1KnLu2lrPuEkrsklrSGPOT9RalT\nl/ZS5n1CJ1QlsaQ15CHvL0qdurSXMu8TSu6SWNIa8pD3F6VOXdpLmfcJJXdJLGkNecj7i1KnLu2l\nzPuEkrsklrSGPOT9RalTl/ZS5n1CpZCSiqQ15CHvL0qdurSXou0TqnMXEakg1bmXRNIa2iL0ag+J\nI/b1ktYST1Y/N0H/9iEGDo7S1zOXrXespnZpC76PimxPmb2g5G5mG4D/DHQA/83dvzvp9W8CXwXG\ngZPAHe7+esqxVk7SGtoi9GoPiSP29RLXEjern5tg4crnGRleAfU5PPn4GA9/73lOHPlsrgm+KttT\nkok9oWpmHcAjwEZgBXCbma2YNNthYI27/0PgJ8C/TzvQKkpaQ1uEXu0hccS+XuJa4mb924cYGV4O\n9S6gA+pdjAwvp3/7UK5xVGV7SjIh1TI9wLC7v+budeAJYFPzDO7+lLuPNp4+B1ydbpjVlLSGtgi9\n2kPiiH29xLXEzQYOjkJ9zoUT63PYPzg69RsyUpXtKcmEJPergGNNz483pk3nTkDHCAGS1tAWoVd7\nSByxr5e4lrhZX89cqI1dOLE2Ru/aubnGUZXtKcmkWuduZrcDa4A/n+b1u8zskJkdOnnyZJofXUpJ\na2iL0Ks9JI7Y10tcS9xs6x2r6V56FGqngAmonaJ76VG23rE61ziqsj0lmdhSSDO7EXjA3W9uPL8P\nwN3/3aT5vgD8F+B33f3tuA9WKWQkaQ1tEXq1h8QR+3rBaoln63y1zP7BUXrXtr5apuzbUz4utTp3\nM+sEjgLrgTeBQeAP3P3FpnlWEZ1I3eDu/zckQCV3EZGZC03uscMy7j4O3A3sA14Gdrr7i2b2oJnd\n2pjtz4Eu4K/N7IiZ7UoQu4iIJBRU5+7ue4A9k6bd3/T7F1KOq20U5SKkpOrjdfqf7WfgjQH6Fvex\n9fNbqXWG3wJPRNKlK1RbqCgXISVVH6+z8KGFjJwZAeDJXz3Jwwcf5sQ9J5TgRVpEXSFbqCgXISXV\n/2z/h4n9vJEzI/Q/29+iiEREyb2FinIRUlIDbwxMOX3/sf05RyIi5ym5t1BRLkJKqm9x35TTexf1\n5hyJiJyn5N5CRbkIKamtn99K92XdF0zrvqybrZ/f2qKIREQnVFuo45IO9t2+rxAXISVR66xx4p4T\n9D/bz/5j++ld1KtqGZEW0806RERKRDfriFOSuxmUpQ6+LHFmTdtBiqI9k3tJ7mZQljr4ssSZNW0H\nKZL2PKFakrsZlKUOvixxZk3bQYqkPZN7Se5mUJY6+LLEmTVtBymS9kzuJbmbQVnq4MsSZ9a0HaRI\n2jO5l+RuBmWpgy9LnFnTdpAiad9SyJLczaAoN+OIU5Y4s6btIFlL7WYdWWl5chcRKSHVuYtMcv4W\neAMHR+nrmd0t8FTHLmWh5C5toX5ugoUrn2dkeAXU5/Dk42M8/L3nOXHks8EJXnXsUibteUJV2k7/\n9iFGhpdDvQvogHoXI8PL6d8+FLwM1bFLmSi5S1sYODgK9TkXTqzPYf/gaPAyVMcuZaLkLm2hr2cu\n1MYunFgbo3ft3OBlqI5dykTJXdrC1jtW0730KNROARNQO0X30qNsvWN18DJUxy5lohOq0hZql3Zw\n4shn6d8+xP7BUXrXzrxapir996U9qM5dRKREQuvcNSwjIlJBSu4iIhWk5C4iUkFK7iIiFaTkLiJS\nQUruIiIVpOQuIlJBQcndzDaY2StmNmxm907x+ifM7MeN1w+Y2ZK0AxURkXCxyd3MOoBHgI3ACuA2\nM1sxabY7gRF3Xwr8R+DP0g60XU18MMHuo7vZ9r+3sfvobiY+mGh1SCJSAiHtB3qAYXd/DcDMngA2\nAS81zbMJeKDx+0+Ah83MvFWXv1aE+oeLyGyFDMtcBRxren68MW3Kedx9HHgP+FQaAbYz9Q8XkdnK\n9YSqmd1lZofM7NDJkyfz/OhSUv9wEZmtkOT+JrCo6fnVjWlTzmNmncAVwDuTF+Tuj7n7Gndfc+WV\nV84u4jai/uEiMlshyX0QWGZm15hZDdgM7Jo0zy7gjxq/fwl4UuPtyal/uIjMVuwJVXcfN7O7gX1A\nB7Dd3V80sweBQ+6+C/jvwF+a2TDw90T/AEhC6h8uIrOlfu4iIiWifu4iIm1MyV1EpIKU3EVEKkjJ\nXUSkgpTcRUQqSMldRKSClNxFRCqoZXXuZnYSeD2FRS0Afp3CcspC61ttWt9qS2N9f9vdY/u3tCy5\np8XMDoUU9FeF1rfatL7Vluf6alhGRKSClNxFRCqoCsn9sVYHkDOtb7Vpfastt/Ut/Zi7iIh8XBWO\n3EVEZJJSJHcz22Bmr5jZsJndO8XrnzCzHzdeP2BmS/KPMj0B6/tNM3vJzH5uZn9rZr/dijjTEre+\nTfN90czczEpdXRGyvmb25cZ3/KKZ/VXeMaYpYH9ebGZPmdnhxj59SyviTIuZbTezt83shWleNzP7\nXmN7/NzMbsgkEHcv9IPoBiGvAtcCNeB5YMWkef4l8Gjj983Aj1sdd8br+4+BuY3f/7jq69uYbz7w\nDPAcsKbVcWf8/S4DDgPdjee/1eq4M17fx4A/bvy+AvhVq+NOuM7/CLgBeGGa128B9gIGfA44kEUc\nZThy7wGG3f01d68DTwCbJs2zCfhB4/efAOvNzHKMMU2x6+vuT7n7aOPpc0T3tS2rkO8XYBvwZ8CZ\nPIPLQMj6fg14xN1HANz97ZxjTFPI+jpweeP3K4C/yzG+1Ln7M0R3pJvOJuCHHnkO+KSZfTrtOMqQ\n3K8CjjU9P96YNuU87j4OvAd8Kpfo0heyvs3uJDoKKKvY9W38t3WRu/+PPAPLSMj3uxxYbmYDZvac\nmW3ILbr0hazvA8DtZnYc2AP8ST6htcxM/8ZnJfYeqlJcZnY7sAb43VbHkhUzuwT4D8BXWhxKnjqJ\nhmZuIvpf2TNmdr27v9vSqLJzG/B9d3/IzG4kuh/zde7+QasDK7MyHLm/CSxqen51Y9qU85hZJ9F/\n7d7JJbr0hawvZvYF4FvAre5+NqfYshC3vvOB64CnzexXRGOUu0p8UjXk+z0O7HL3c+7+S+AoUbIv\no5D1vRPYCeDuPwMuI+rBUlVBf+NJlSG5DwLLzOwaM6sRnTDdNWmeXcAfNX7/EvCkN85clFDs+prZ\nKuC/EiX2Mo/HQsz6uvt77r7A3Ze4+xKicwy3untZ764esj//lOioHTNbQDRM81qeQaYoZH3fANYD\nmNnvECX3k7lGma9dwL9oVM18DnjP3d9K/VNafWY58OzzLURHL68C32pMe5DojxyineGvgWHgIHBt\nq2POeH3/F/D/gCONx65Wx5zl+k6a92lKXC0T+P0a0VDUS8AvgM2tjjnj9V0BDBBV0hwB/mmrY064\nvj8C3gLOEf0v7E7g68DXm77fRxrb4xdZ7c+6QlVEpILKMCwjIiIzpOQuIlJBSu4iIhWk5C4iUkFK\n7iIiFaTkLiJSQUruIiIVpOQuIlJB/x8Ris5p2f4nKwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5f1d593390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def Normalize(data):\n",
    "    '''\n",
    "    对数据进行归一化处理\n",
    "    '''\n",
    "    n_dimension = len(data[0])\n",
    "    new_data = np.copy(data)\n",
    "    \n",
    "    for d_idx in range(n_dimension):\n",
    "        minValue = np.min(data[:,d_idx])\n",
    "        maxValue = np.max(data[:,d_idx])\n",
    "        new_data[:,d_idx] = (data[:,d_idx] - minValue)/(maxValue - minValue)\n",
    "    return new_data\n",
    "\n",
    "data = Normalize(data)\n",
    "drawData(data, target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 距离计算公式 - 欧式距离\n",
    "\n",
    "实际上，距离计算的方法有很多种，例如欧式距离，汉明距离等等，详细的距离计算方法，大家可以参考一下博客[各种距离算法](http://blog.csdn.net/a_12_123_1234_12345/article/details/78857062). 这里我们只讲解欧式距离。\n",
    "实际上，我们在平面几何中， 计算距离采用的就是欧式距离。\n",
    "\n",
    "![1353398777_7638.png](./IMG/1353398777_7638.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在2D平面上， 点$(x_1, y_1), (x_2, y_2)$之间的距离计算公式为：\n",
    "\n",
    "$$d = \\sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2}$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.20833333,  0.13678969,  0.16197089,  0.17179607,  0.30460711,\n",
       "        0.13888889,  0.29000851,  0.29000851,  0.33592741,  0.50689688,\n",
       "        0.39771725,  0.41036908,  0.45069391,  0.08333333,  0.47466687,\n",
       "        0.22438187,  0.58597286,  0.2003084 ,  0.17179607,  0.34359214,\n",
       "        0.30838338,  0.186339  ,  0.2003084 ,  0.38888889,  0.21561354,\n",
       "        0.12804923,  0.25609846,  0.25      ,  0.48848626,  0.37060178,\n",
       "        0.35600016,  0.17179607,  0.41759157,  0.3167154 ,  0.13888889,\n",
       "        0.22438187,  0.33592741,  0.51219691,  0.32542707,  0.16896563,\n",
       "        0.21154925,  0.47222222,  0.17179607,  0.22609473,  0.08784105,\n",
       "        0.21154925,  0.1001542 ,  0.06944444,  0.50993222,  0.30555556,\n",
       "        0.22438187,  0.12804923,  0.37371178,  0.28088539,  0.30460711,\n",
       "        0.06944444,  0.30555556,  0.3167154 ,  0.08784105,  0.50993222,\n",
       "        0.37060178,  0.30460711,  0.33592741,  0.22222222,  0.1001542 ,\n",
       "        0.372678  ,  0.17179607,  0.37371178,  0.21017703,  0.3167154 ,\n",
       "        0.37060178,  0.37371178,  0.34805456,  0.44876373,  0.18685589,\n",
       "        0.35053971,  0.33592741,  0.41759157,  0.2795085 ,  0.08333333,\n",
       "        0.25      ,  0.37060178,  0.26352314,  0.34359214,  0.44292274,\n",
       "        0.55901699,  0.27357938,  0.40277778,  0.3167154 ,  0.372678  ,\n",
       "        0.36350701,  0.32304732,  0.16197089,  0.37909289,  0.38888889,\n",
       "        0.22438187,  0.26352314,  0.33013512,  0.17568209,  0.45218946,\n",
       "        0.0931695 ,  0.42491829,  0.16666667,  0.22609473,  0.35053971,\n",
       "        0.08784105,  0.42491829,  0.21154925,  0.18685589,  0.44876373,\n",
       "        0.13888889,  0.31211396,  0.2003084 ,  0.29691053,  0.34722222,\n",
       "        0.33592741,  0.21695138,  0.27357938,  0.186339  ,  0.16666667,\n",
       "        0.08333333,  0.41874482,  0.21154925,  0.22438187,  0.22609473,\n",
       "        0.1001542 ,  0.05555556,  0.11111111,  0.41759157,  0.41666667,\n",
       "        0.13888889,  0.125     ,  0.4837243 ,  0.16197089,  0.47951879,\n",
       "        0.186339  ,  0.34805456,  0.186339  ,  0.29691053,  0.33013512,\n",
       "        0.19444444,  0.58151171,  0.30046261,  0.1502313 ,  0.28497617,\n",
       "        0.16724437,  0.186339  ,  0.47140452,  0.25038551,  0.08333333])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def EuclideanDistance(data, vect):\n",
    "    '''\n",
    "    计算一个点与其他点集的距离，输出一个距离列表\n",
    "    '''\n",
    "    return np.sqrt(((data-vect)**2).sum(axis=1))\n",
    "\n",
    "EuclideanDistance(data, np.array([0.5, 0.5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K最近邻算法的实现\n",
    "\n",
    "为了讲解KNN，同时也因为KNN模型足够简单，所以阿凯打算先讲解如何自己动手DIY KNN算法。\n",
    "前面讲解了获取一个点距离数据集的所有距离的集合，所以在KNN函数中，需要做的就是需要提取前K个最小距离的数据并统计他们所属的类，最后输出最多的类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.20833333  0.13678969  0.16197089  0.17179607  0.30460711  0.13888889\n",
      "  0.29000851  0.29000851  0.33592741  0.50689688  0.39771725  0.41036908\n",
      "  0.45069391  0.08333333  0.47466687  0.22438187  0.58597286  0.2003084\n",
      "  0.17179607  0.34359214  0.30838338  0.186339    0.2003084   0.38888889\n",
      "  0.21561354  0.12804923  0.25609846  0.25        0.48848626  0.37060178\n",
      "  0.35600016  0.17179607  0.41759157  0.3167154   0.13888889  0.22438187\n",
      "  0.33592741  0.51219691  0.32542707  0.16896563  0.21154925  0.47222222\n",
      "  0.17179607  0.22609473  0.08784105  0.21154925  0.1001542   0.06944444\n",
      "  0.50993222  0.30555556  0.22438187  0.12804923  0.37371178  0.28088539\n",
      "  0.30460711  0.06944444  0.30555556  0.3167154   0.08784105  0.50993222\n",
      "  0.37060178  0.30460711  0.33592741  0.22222222  0.1001542   0.372678\n",
      "  0.17179607  0.37371178  0.21017703  0.3167154   0.37060178  0.37371178\n",
      "  0.34805456  0.44876373  0.18685589  0.35053971  0.33592741  0.41759157\n",
      "  0.2795085   0.08333333  0.25        0.37060178  0.26352314  0.34359214\n",
      "  0.44292274  0.55901699  0.27357938  0.40277778  0.3167154   0.372678\n",
      "  0.36350701  0.32304732  0.16197089  0.37909289  0.38888889  0.22438187\n",
      "  0.26352314  0.33013512  0.17568209  0.45218946  0.0931695   0.42491829\n",
      "  0.16666667  0.22609473  0.35053971  0.08784105  0.42491829  0.21154925\n",
      "  0.18685589  0.44876373  0.13888889  0.31211396  0.2003084   0.29691053\n",
      "  0.34722222  0.33592741  0.21695138  0.27357938  0.186339    0.16666667\n",
      "  0.08333333  0.41874482  0.21154925  0.22438187  0.22609473  0.1001542\n",
      "  0.05555556  0.11111111  0.41759157  0.41666667  0.13888889  0.125\n",
      "  0.4837243   0.16197089  0.47951879  0.186339    0.34805456  0.186339\n",
      "  0.29691053  0.33013512  0.19444444  0.58151171  0.30046261  0.1502313\n",
      "  0.28497617  0.16724437  0.186339    0.47140452  0.25038551  0.08333333]\n"
     ]
    }
   ],
   "source": [
    "# 拟定一个测试点 坐标是0.5 0.5\n",
    "vect = np.array([0.5, 0.5])\n",
    "# K设定为5\n",
    "K = 5\n",
    "# 获取距离列表\n",
    "distance = EuclideanDistance(data, vect)\n",
    "print(distance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([126,  55,  47, 120,  79])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 获取排序从小到大对应的编号 并截取前K个index\n",
    "kn_idxs = distance.argsort()[:K]\n",
    "kn_idxs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 1 1 1]\n"
     ]
    }
   ],
   "source": [
    "import collections\n",
    "\n",
    "# 获取前K个邻居的标签\n",
    "kn_labels = target[kn_idxs]\n",
    "\n",
    "print(kn_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{1: 4, 2: 1}\n"
     ]
    }
   ],
   "source": [
    "# 对标签进行统计\n",
    "kn_labels_stat = dict(collections.Counter(kn_labels))\n",
    "print(kn_labels_stat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    }
   ],
   "source": [
    "# 获取数量最高的标签\n",
    "predict = max(kn_labels_stat, key=lambda x: kn_labels_stat[x])\n",
    "print(predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def KNN(data, vect, K=3):\n",
    "    '''\n",
    "    K 邻近算法的实现\n",
    "    '''\n",
    "    distance = EuclideanDistance(data, vect)\n",
    "    kn_idxs = distance.argsort()[:K]\n",
    "    # 获取前K个邻居的标签\n",
    "    kn_labels = target[kn_idxs]\n",
    "    # 对标签进行统计\n",
    "    kn_labels_stat = dict(collections.Counter(kn_labels))\n",
    "    # 获取数量最高的标签\n",
    "    predict = max(kn_labels_stat, key=lambda x: kn_labels_stat[x])\n",
    "    return predict\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "# 拟定一个测试点 坐标是0.5 0.5\n",
    "vect = np.array([0.5, 0.5])\n",
    "# K设定为5\n",
    "K = 5\n",
    "\n",
    "KNN(data, vect, K)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测效果检验\n",
    "\n",
    "划分训练集跟测试集, 使用80%的数据作为训练集， 剩下的20%作为测试集。\n",
    "\n",
    "**注意：KNN没有预先训练模型的过程**， 这个模型比较**懒**, 每次判断一个点类型的时候，是将这个点跟训练集中的每个样本一一比较，然后选取前K个比较邻近的。 类似这种的学习方式被称之为**Lazy Learning 懒式学习** '有点像是平时不努力学习功课，到了考前才开始临时抱佛脚的感觉。'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率：　0.80\n"
     ]
    }
   ],
   "source": [
    "# 训练集\n",
    "X_train = data[:120]\n",
    "y_train = target[:120]\n",
    "# 测试集\n",
    "X_test = data[120:]\n",
    "y_test = target[120:]\n",
    "\n",
    "def evaluateModel(X_train, y_train, X_test, y_test, K=1):\n",
    "    y_predict = np.ones_like(y_test)\n",
    "    n_test = len(y_test)\n",
    "    \n",
    "    for idx in range(n_test):\n",
    "        vect = X_test[idx]\n",
    "        predict = KNN(data, vect, K)\n",
    "        y_predict[idx] = predict\n",
    "    \n",
    "    # 准确率\n",
    "    accuracy = np.count_nonzero(y_predict == y_test) / n_test\n",
    "    return accuracy\n",
    "\n",
    "\n",
    "accuracy = evaluateModel(X_train,y_train, X_test, y_test,K=5)\n",
    "print(\"准确率：　%.2f\"%(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 可视化\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXt4HNV9Pv6emdXepJVkSb5ItizJtqyLbVmyLbs2BgOO\nISam0BKSAG1C4pa2v28uTWlJ89CEW/mloaTf0jQhQELInUICBAPGUIcYGyNj3WzZsuS7JEs2knVd\n7ezsamfO94+PVrurXWl3pdXurDTv8/iR952zZz5zzpmzM++88zmMcw4dOnTo0DG7ICQ6AB06dOjQ\nEXvok7sOHTp0zELok7sOHTp0zELok7sOHTp0zELok7sOHTp0zELok7sOHTp0zELok7sOHTp0zELo\nk7sOHTp0zELok7sOHTp0zEIYErXjnJwcXlhYmKjd69ChQ0dSoq6u7irnfH64cgmb3AsLC1FbW5uo\n3evQoUNHUoIx1hZJOV2W0aFDh45ZCH1y16FDh45ZCH1y16FDh45ZCH1y16FDh45ZCH1y16FDh45Z\nCH1y16FDh45ZCH1y16FDh45ZiLCTO2PsecZYN2PsxATbGWPsvxhjZxljxxlj62Ifpg4dOnToiAaR\nXLm/AOCTk2zfCaB49N99AJ6eflg65hwUBXjjDeCxx+ivoiQ6Ih06khph31DlnL/PGCucpMhtAH7O\naaXtGsZYJmMsl3N+OUYx6pjtUBTg5puBI0cAhwNITQU2bQL27QNEMdHR6dCRlIiF5r4YQIff50uj\nnA4dkWHvXprYh4cBzunvkSPE69ChY0qI6wNVxth9jLFaxlhtT09PPHetQ8toaKArdn84HEBjY2Li\n0aFjFiAWicM6AeT7fV4yygWBc/4sgGcBYMOGDXxae+UcGBoCJAmwWoH0dIAxnU9GfuVK4vwn+NRU\noLgYuHx56vUD2jvWJOA1Fk7S84lCLCb31wF8mTH2IoBNAAZnXG/nHDh1Crh4ERAEQFWBwkKgtBRo\nadH5ZOOtVprIW1oAl4sm9lWriD92bGr1FxTQWGlr09axapznpWU41cK0Ek7S82VliZvgw07ujLHf\nALgeQA5j7BKAhwCkAADn/EcA3gJwC4CzACQAX5ypYMcwNEStuGCB7yfz4kUgLU3nk5HPzQWeeQb4\n4AOSaG64AcjJIX6q9Tc301gpKtLWsWqcH0pbjIsXM7QSTtLzixcDGRnhJrSZQSRumbvCbOcA/k/M\nIooEkkQt6HAAsgyYzfS5r0/nk5lft44ugXJzSY6ZTv1uN42VRB+ThnnOBAw72CjNkMYESH0yGMvQ\nUphJzUuShid3TcJiodttWSarnKJQa5aXAzU1Oq/zdF8MAD09iY9FgzwvL0d7TRc6ZA5BZFAVjnyz\nivRyM9q0E2bS85s3J26aTM7JfSLwCZ7R6rzOazGWBPJDPB1tKMQiXASZ5lS0oRAFPF1LYSY9n0gk\n5+TudNIDM6vVdw8kScDAgM7rPPGXLtEZl5+f+Fg0yEsDbkgFZXBYF0OQJahmKyQpHf0DTEthJj3v\ndAKZmYmZJpMzcZjVSrfd3p9LzulzVpbO6zz9TUkBjEZtxKJB3pplhqIyDPIMXEEuBnkGFJVpLcyk\n561WJAzJeeVus5GodeCAT+CqqgLy8ujFF53X+bVr6YmWFmLRIG/Ls0HRTjizlrfZEjdNJufkbrdT\nC27bRr5ok4keU3d16bzOE9/dTWNFC7FokLd32SGKGVoJZ9bydrvulokOkkQtabP5fhplmfxIOq/z\nAJ1tnGsjFg3yUp8MUczQSjizlk+kFTJ5NXdFoZ/Fq1fpr6KQ8KXzOm+3AwYD6e5aiEWDvDXLrKVw\nZi2va+7RQtfctcdXVNDbpS+9RFfMpaX0V9fcNcnrmruuuWsTuuauLd5gAO6/HzhzhrxfZjNQUgI8\n+STdm+qau+Z4XXPXNXdtQtfctcUfPAi0ttI2gCb41lagqYmu4HXNXXO8rrnrmrs2oWvu2uKPH6fL\nFX/IMk3uiYpT19wn5XXNPT68rrlHC11z1xYvy3Qf6r1yB2hidTrpO7rmrjle19x1zV2b0DV3bfGb\nN1OKXa/mbjKRHLN7t665a5TXNXddc9cmdM1de/zjjwPnzgF1dXTJsmULlenp0TV3DfK65q5r7tqE\nrrlrjweA228HPvMZoLIS6O9PbDy65j4pr2vu8eF1zT1a6Jp7dPzICGUxqq0FPB5aBywlRXtxxpLX\nNXddc9cAr2vu0ULX3CPnJQn41rdIE/eWKS8HHnvMtyi1FuLUNXddc5+FvK65Rwtdc4+cb2ykxcS9\nThZZps+nTwPXXqudOHXNXdfcZyGva+7RQtfcI+fr6wMtigB9bmjQVpy65j5tnisq7HaOnquA3c7B\nFXVCXtfc48Prmnu00DX3yHnvwhXeBaMB+qyqtF0rccaan2OaO89bjHONw+g/cBFMFMAVFfOqCrEs\nbwnOh+CL8tJ1zV3X3DUIXXOPnK+uJr19vOZ+11265j6L+KGuYZwWy5C7bTEElwTVZMVpRzqMXWxC\nXkPhz1pe19yjha65R8c//TSwdy9NeAsWADt30natxRlLfo5p7lKfDEHMgGqjfwAgjBYXRBaS11D4\ns5bXNfdooWvu0fGSBGzYAHzjG/RXkrQZp6656xr6LON1zT1a6Jr73OQXLgSeew744ANg8WJg2TJg\n/fpZqbnrGvrs4HXNPVromvvc44eGgO3bKb2B/7OD9etnpeaua+izg9c192ghSfSWJWP0mTH63Nen\n87OVr68PtHV6/fr19cDSpcHlR0Z8/0907GF4LogYZjbIsMHMgDSBNHQmZGCIZUBGBswMYKPVMIGF\n5DV2WDovJFZzT87J3WIB2troBPfeA5nNdCVXU6Pzs5EP5dd3Oon/4heDy6sqlenpSXzsk/C8vBzt\nNV3okDk9+FQ48s0q0svNaNNOmDo/RX7z5vhPj14k5+Q+ETjX+dnK5+fTj7ok+Tijkfho6tHSMQEY\n4uloQyEW4SLI36CiDYUo4OlaClPnp8gnEsk5uTudQEEBPYqWZfqJlCRgYEDnZyu/YgUlPvOu+uTV\n3HfuDF3+0iU64/LzEx/7JLw04IZUUAaHdTEEWYJqtkKS0tE/wLQUps5PkXc6gczMxEyTEVkhGWOf\nZIy1MsbOMsb+OcT2pYyx9xhjDYyx44yxW2Ifqh+sVrrt9v5cck6fs7J0frbyjAEvvgg88ADJMN/8\nJvDEE8SHKp+SQlf2Woh9Et6aZYaiMgzyDFxBLgZ5BhSVaS1MnZ8ir2krJGNMBPADADsAXAJwlDH2\nOue82a/YvwB4iXP+NGOsHMBbAApnIF6CboWcm3x+PrljBIH+HTqU9OkH9NS7s5vXuhVyI4CznPPz\nAMAYexHAbQD8J3cOwCsSZgDoimWQQdCtkPHlN28GfvYzcqe0tJAcosU4k9AKqafend281q2QiwF0\n+H2+BGDTuDIPA3iHMfYVAKkAPhGqIsbYfQDuA4ClS5dGG6sPevqB+PGKQnlo7HbiGxqAN98EfvlL\nbcU5nk+S9AN66t3Zzc+G9AN3AXiBc74EwC0AfsEYC6qbc/4s53wD53zD/Pnzp743Pf1A/Pjf/MY3\nsXtht/t4rcQ5nk+S9APWLDNUhYPb7cDVHnC7HarCtRamzs/S9AOdAPL9Pi8Z5fyxG8AnAYBz/iFj\nzAwgB0B3LIIMgq65x48/eTJ0HzQ3ayvOJNXc0/PSUNh4DhcP9I/53Aur5mFx3nI0NjKthKnzU+S1\nrrkfBVDMGCsCTeqfA3D3uDLtALYDeIExVgbADKAnloEGQNfc48evWgU0NQX3QXm5tuIczyeJ5s66\nOlEmnsbibbmQXAKsJhXpjtMY6poPXYtPfl7Tmjvn3MMY+zKAfQBEAM9zzk8yxh4FUMs5fx3A/QCe\nY4x9HfRw9V7OZ9DWr2vu8ePvuos0dn9pxmYjXktxjueTRHNHXx+YKCDDpiLDNvpWrSzoWvws4TWv\nuXPO3+Kcr+ScL+ecPz7KfXt0YgfnvJlzfg3nfC3nvJJz/s5MBq1r7nHkRRH47W+Bu+8G1q0DPvc5\neuVfFLUV53heg5q7nsJ37vFa19y1B11zjy9fWwuUlQErVxJfWKhr7lHyegrfuclrXXPXHnTN3ce7\n3cD779NkW15OGrkW4wzHSxJw7Bjw0ktAby+QnT2rNHc9hW/y8tMZmnY7kJZGC6E1NNCE778Q2kwi\nOSd3XXMnvrOTZBJFIe7KFbp0OHhQW3GG461WSitw4gTxr7wClJQAzzwzazR3fRm85OSnOzTtduCO\nO4AjR2iyT00FNm0C9u2b+QleX2YvmfnvfMc3sXvhz2slznD8/v3kyHE6aUJ2OulN2P37k1Jz1zX0\n2cNPd2h+8AFN7MPD9P3hYfq8d+/MT5PJeeWua+7EnzgRun1OnEiu9nn3XbrM8YfLRbyiTK3+BGnu\nE2nruoaenPx0h2ZrK12x+8PhoLK7dsVuSgyF5Jzcdc2d+NWrgQsXgttn9WptxRmO37ED+PDDwLPI\nZCJ+3bqk0twn09a12PQ6P7NDs6SEpJjhYd/3U1OBysqZmRr9kZyTu665E//NbwJvvRUozYgi8YOD\n2okzHL99O/Dqqz5h02yms2L7dp8wmSSa+2TauhabXudndmhecw1p7OM19507MeNIzsndX3P3/kx6\nha/OzrnDL15MOc6fftrnlvm7vyO+r087cYbjJQl4+GGyJDQ3AzfeSJYESZp6/YbRoT2DsfPOLgzb\nOWQXg9nEkebV1jXYxDqfmKFps9HD0717SYqprNTdMpND19x9vNEIXHcdcMMNxBuN2owzUv7OO4Fb\nbqE7Eg373HVtfe7xUxmaNhstPbBr18xr7OORnJO7rrn7eIAm9FOn6EUjIPp6OjqAujr6W1gIbN1K\n95Te8lu30mP/tjZaXmbNGu21w3h+hjV3XVvX+Uh4u13DuWU0CV1zJ76nB3joIXok7y8I/vjHkdfj\ncJBX3rs2qdlMD2QfeojKA6The0VHkwl4+23g29/WTjskQHPXtXWdj4TXfG4ZzUH3udPn48fJdDve\nhHv8eOT1fPSRb+L21tHURHxWlm+7dx+yTJ8/+kg77RAjn7ue+0XnY83ruWWiha65E9/eTlfb/nC5\nSF7Jzo6snrNnaeL2hywD585R+VDbnU7aLgjaaIcYaO567hednwneexWfCCTn5K5r7sQvXUr/H2/C\nzc+PvB5VBSwWun/0wmwGli+n8itW0Hb/Cd5ioe1aaYcYaO567hedn22ae3LKMpJEV42M0WfG6HNf\n39ziKypIY7dYiLNY6HNFReT1VFeTxm4202ezmSyV1dVUvrqaHtT6b1+9mnittEMofmSE/oUoywUR\ndmZDD3JgZzZwQYTUJ4MJwJBRwBWLAUNGAWy0aiYwDLEMXEEuhlgGmMA0dag6r13e/5op3kjOK3eL\nhZwbsuy7B/JOSjU1c4u/7z5KN3f5MpCbS/eCOTnA0aOR1/Ob3wDf+x697ZqfTz8Qly6RK6amBrjn\nHnpo29EBFBUB998PvPOOttphPK+OLnzR0xNQlpeXo72mCx0yH1vWLt+sIr3chLbmdshSB0RRgKKo\nMLvyUV6+FDU1TBOHpPPJx2/enLhpMjkn94kw0eJPs5kXBErzW1FBI0oQoq9HEGgiLy72jUz/8oJA\nV+veK3hhghs+LbbPOAzxdLShEItwEXTjqqINhSgwAchsA64uAkQACoCcNnBjJoDg+2oNHZLOa5hP\nJJJzcnc6gYICehTttQBKEjAwEFveZAIOHaK7hPx8SuYcj/3683Y78PrrdCVeXk4PClNTZ36/yc5f\nukRnXH5+QFlpwA2poAwO62IIsgTVbIUkpaPfcQUFxRKsyxyQnQLMFhWSKGFgWEZBQYYmDmm28Q4H\nvflZV0fLEixbRg8gtRbndHinE8jMTMw0mZyau9VKt93en0vO6XNWVuz4kRHgn/6J0uf+9KfAd78L\nfO1rxM/kfv15jwf4l38hn/mvfgU89hjw4IPEz+R+ZwOfkgJuNMHOU0lb56ngKicLowoMGkhbHzQI\nUFQgy2aFqqrgTAFEDzhToKoqstLNmjmk2cR7PDSUH3uMhvY3v0lD3ePRVpzT5XUrZLSIhxXyuefI\nz+120z5lmcoePUoe6nj4qZqbfS8oAXQZ0NwM/OQn9OKR1nxfGuL52kqcY8vRf6B9nIXRBqXtHA6c\n7IdoYFA8HFVl85CXtQyN5wpx4PzFMc29alkh8pano1F33cac/8lPaCh7Ty9JoqH+k5/QDapW4tSt\nkPFGPKyQFy7QVbo/JAm4eBHYvTs+fqrz54N97CMj9EBZFLXn+9IQP9TtwmkUIndbfqCFsW8I4oLT\n2JaTC5cswGRW4RBOo6t3PsS+MmxbthguVYJJsMLRl44u3fI4I7zZHHx6uVw0tKurtRNnMlshk3Ny\nj0f6geJiGoH+/m6zmXzf8XqHuaIi2MduNvtyu2jtXWsN8VIXh8BdUG3ZgekB7BJEUYDNpsKWQY4a\nWRLQNyRDFDNgM2bANvoAVdabeMb4iorg08tkoqGdk6OdOPX0A/FGPNIPVFUF+7vXrSM+Xu8wr1lD\ntkSr1RdDSQnxM7nfJOTHpwiwGEagpJiDi9qsUEbLXh0tqyikrWvskGY17x3a3tPLavUNbS3Fqacf\niDfilX5g1y4SADs7Kev+v/wLJWeOp3D30ENkPXz5ZXpCU1pKDh4tCowJ4kOlDshYWwgPswZXkZWu\na+sa4A8dAv7szygVkiBQOl1VJV5Lceqae7wRr/QDN9wAbNni4z/+ODHC3Zo19BA3NVUbQqLG+FCp\nA451p4ODhaiC6dq6hvjrrvMN8aamxMeja+6JxlxK+Ts0BPzrv1IWxupq4Etfovzt3vKKQl6yY8fI\nA3/rrcH1eO9y3nwTWLiQloKJdZwOB/Daa0BtLclXW7ZQmTi0W6j0uyNdZEcLXQXTtXWdn/Wae3JO\n7nNlmT3G6NV/7+q6x47Rsnq/+x2VT0sL3H70KG1vaPC1jyQB3/oW+c68+3r9dTIYe9txunEaDGRa\nPnOGnpD99rckoD75ZFzajZa247DLw3ApMkyiGQZDGgAEcIqShqwspomu1fm5weuae7SIl+aeaP7d\ndwOXTQfo8+OPU36XF14Ivf2FF4CNG6mes2eD/fonTlA+mRUrZoUf35aXFuRdX1uSCQbgwMmBQD97\n3nI0NrKEd63Ozw1e19yjRbw090Tzr7wS+viHh6nce++F3v7HP1LGom3bKO/6eEPxyAg9xdq2bVb4\n8e19nUHe9W5PIwBg27Urgv3sYkbCu1bn5wavec2dMfZJAE8BEAH8mHP+byHKfAbAwwA4gGOc87tj\nGGcg5ormvn49SSzjsW4dlVu1Cjh8OHh7ebmvnnXrSCYZ79f3XlbMAj++ZO/DeO96V+8IOOewZUzg\nZ9dol+v87OI17XNnjIkAfgBgJ4ByAHcxxsrHlSkG8E0A13DOVwH4+xmI1Ye5sszenXeSru6PtDTi\nFQV44IHQ2x94wFfPypXBfv2yMuJjFWcc/fhcUWCX7ehxXIVdtoMrCqy2LCiKArvLjqvSVdhddhhg\nQIqQEsApiqL72XVe97n7YSOAs5zz8wDAGHsRwG0Amv3K/DWAH3DO+wGAc94d60ADMFc0d0WhpGWN\njcCePZRe7oYbgA8/pPKFhbT9+9+nLIhLlgBf+Qrxzc2+enbtAm65hdYU9Uoy9fWxjTMOfnyel4dz\nbY3oP3kAzCCCexTMK6tCUVYelL5GHLh4AKIoQlEUrF24FgwsgKvKrUJetk33s+t83Hita+6LAXT4\nfb4EYNO4MisBgDH2AUi6eZhz/nZMIgyFuaK5e/n77iMNPZTP3WIBHnkksnq8ZuKZ8svPsB9/qK8L\npxeIyM3ZBkF2QTWbcFpwwOjogshEbCvcBpfigkk0oXuYri/8OYfbga5eO3TNXed1zT26eooBXA9g\nCYD3GWNrOOcD/oUYY/cBuA8Ali5dOvW9xUtzt1rp57i1FVi0iP5FU8+VK+RBP3uWPOhf+lLsfe6A\nL8aSEsqJE2ufu8nk89KvWAH8/d8nRMCU7H0QRBGqzQY1g3hBktEn9UEURdhMNthAfNdwF2nufpxj\nxIHXTr2O2isXsS63ClsW7ITIxKhCCdeUPT1k9b9yhbpjyxZtasE6P/s190gm904A+X6fl4xy/rgE\n4AjnfATABcbYadBkf9S/EOf8WQDPAsCGDRv4VIOOi8+9vR34u78L9Ie/+y7p2ZHU43QCN90EDA5S\nzEePAv/zP8DPfhY7n3tGBnDvvcDp0zSSzGbS0l991VfPdH3uFy4An/kMbfMex1tvkRwUZ9Ow1ZYF\npb8Tdpd97GpcURRkWbPQORTIG2AABAa7ncPlYjCkePDggQdxpvcsnB4Jv+1KRUnaJjxZuQ+KIkYU\nSrimdLsp5X9dnW97eTkpVlrzX+t8fHita+5HARQzxopAk/rnAIx3wrwG4C4AP2WM5YBkmvOxDDQA\n8dDcp5vP/dQp38Tuhd1OE/ytt8bG5378eLC/vLWVeMZi43P3/pj5Y3CQjqOsLK4Cpi2Etl6VW4W8\n1Dw08vGaeyVY/3IcuNgOURTQPFSD1qtnICu0YrFTHUbz0BH85P29+FzVrohCCdeUHg/9/nq7w7v9\n6FF65KElLVjndc0dnHMPY+zLAPaB9PTnOecnGWOPAqjlnL8+uu0mxlgzAAXAP3HOe2cs6nho7tPN\n5z6RR72lJXY+9z/8Idhf7nIBBw/Sg9dY+NxPnZr4OP72b+MqYNr7grV1h9uBrlCae68L6C7EtmX5\ncKkSzl84CJfiDGwGOGApbIQo7ooolHBN+eMf03fGb794UXtasM7Pfs09opS/nPO3OOcrOefLOeeP\nj3LfHp3YwQn/wDkv55yv4Zy/OJNBQ5LojGKMPjNGn/v6YsevWEEyhz/MZmD58sjqWbUqdOzl5ZHH\ns2ZN6DpWr6Zy3nVe/WEyAUuX+uoJdxzh2mGy45jJ9g/BS/Y+CIIANsozxiAIAvqkYH7ElYIR1QXG\nBUAxoNi2FiYhNbCpBCtWZK6EIPCIQgnXlJNtn+Gm0XmN8pKEhCE531C1WGjRaln23QOZzTTh1NTE\nhl+wgKyFbW10H24y0X3WggX0YlG4erZsIW3aX5qxWIifNy+yeNasIdHOf4RYLMS3tZHF8Yc/pLsM\nt5setBYUEP/OO1RPRgYdR3u7r8ySJcQ3NIRvhy1bKCGY/0tQGRnER9IOMeQtVeVoa6+BrMgQBRGK\nqsAsmlGeU46ajkBelW2APRc96IUoCjB5FqLAVIGOkeOQPA4YmRlLraUwjSxEm6Md5eVLUVPDJg0l\nXFMuWEBDxKu5e7cvWBB5l+v87OI3b47ftDgeyTm5TwTv6rSx4AWBZIfWVqCjAygqAv7hH0gHj6Qe\ng4EcLl//OiXUWrYM+MQniI80HoOBFuY+fZom64IC4JOf9F0eCALw1a/SzHL5MpCbS7OL4HdD5j2O\nc+dotMkyXUoKAo3AcO1gMACPPgr87/9SmoHiYuD//l+ShLyrGU/WDrHkEZrnoXjTEPi8i8DgckAE\nBAX46qp/hSn3In5f/wEWeqqxOu06cEkEctrAjZkAgu+f/UMJ15SCAPz858B//Af93ubnk2NGEGa+\naXRem3wikZyTu9NJE53V6nOJSBIwMBB7ftUqH2+3R1ePJAFf/vL049m6lTx3ocoXFVGM4eopK6PJ\n//LlqcWzZk3gccWr/f14p2MABfMKYE2xQvbIMBvMkEYkDMjB/KWhS+C2PuSbFkF2CjBbVEiiC7lp\nVcjblgurshCy0z3KSxgYllFQkBFRKJM1pd1ON05xbhqd1yjvdNK7h4lA8i6zp6q+n0vO6XNW1uzj\nR0boyvxXv6I8MooSWH5khHjv9pERbcUfQ95qy4KqquCjPOccqqoiyxrMp7AUGMUUcNMgYLsCbhqE\nqiqjZZVgPt0MVeXgBjtg6QE32KGqPOrQ02wKfr7/I3z9qT/i5/s/gmtEiXuTud3A888D//zP5FiV\nZc104ZzjtW6F1B7mSvqBkRHSu8+fp0fvKSkk4v7FX1BSsYULyXjd0uITgUtLSQ7SQvwJtUJOkH4g\nRFlKS5CGxvTAtMHRpghetUbBxluPof9cOeC2oMHoxCu/PYZfPLkWeXliXJps/35Ko++1Yx4/TpmX\nv/51n/dag107a3lNWyE1ibmSfqC+3veCEkATeFcX3fOJIvDrX9N2r/Ha7abPv/41PcVLdPyJtEJO\nlH4gRFnig9MGR5si+Jk36kYn9tFkbu40yJ0r8bvDddi4YWNcmqy93TdcvHA4iL/77oR34ZzjZ0P6\ngfhirqT8bW8P7WNvb6fLgqNHQ2+vraUXpRIdf4x5Su0bmGZAViJPPzBRWR8fmDY42hTBp85KgNsS\n2B9uC1rOSXFrsoleSzh1CsjJSXgXzjle0yl/NYm5kvJ36dKJfeyKQrlmQm3fsEEb8fvxoVL1RsuH\nSu3rTT8QccrfEGUn5aNIEVy2wgoYA1+UgtGJ0uXWuDV9WVnoU6asTDNDYU7xuuYeLeaS5r5yJXnv\nJIk097w8uiRQFLrP/s//DNTcV64kft++xMc/yk+UqndZXh7OR8GHSu0bM819Qi0+8hTBd2xZj31v\nHUP/uZV0BW90wrz4NO7Ysj5uTb90qc+m6UVqKvFHjyZ8KMw5Xtfco8Vc0dxNJnpw2ttLvvLycsou\nabVS+Y8/pqxVJ0/SBF9aSrbIjz/WTvxhUvVGy8+c5j4BH1WKYBEf7VmLbz1dh9MXJJQut+ILN63H\niEuMW9Nv307L5/72t5TnZts24Prr6SUqDQyFOcfrmnu0mCuau5evqiJj9fz5vjbwljcaKfvkTTcR\n39OjufgnS9UbLT+zmnsIPspl+YbtIr7wiY0BXZWILvnqV2m/ixZR+mGNDIU5x2s95a/2EEnKX0mi\nS5eTJ+lyJjs7+vydAwPABx9Q5qf8fEqvOz4l789+Rg84ly4FvvhFMrhGWn8kvNtNPvf9++mq3Hvl\nHqv6p8jzzk4My3bIigtm0YQ0RQGbgI8mVe90+AHnAD7o+AAXBy9ivmU+qnKrpl93phmdCoddHoZL\nkWESzVASJIxpAAAgAElEQVSUNGRlsSk1ZW+/gp+9U4dTZyWULLPisb9bD6VTjHuXZ2RQZuiODlq4\n65prohv6BqOCD3v3YtjdgDR7Fa4z7YTHLUYVTyxOUa3zuuYeLcJp7u+9B/zyl7T03MgI8Pvf0+v/\nt99OunUkAlpdHS1Z19lJZ5vZTNkWv/IVcqNwTqkBvOLm4cPkSf/GN2jSj4fPPUk09Gi08qnydUod\nvvL2V9A51Am36oZZNGNl9krcXnI7Ugwp09Dcp+9/9/If1Sr49FeOQe70+eD3vXUMz/7rWtTWinHr\n8ooKYPduWq1Rlunmb/Fi4Hvfi2zoM0HBC+rNuCwcgUt1wCSk4hdnN+FeYR+4KkYUTyxO0WTgdc09\nWoTT3M1m+r/X/z08TP5vsxlYty4yAa2xke5n/ZN3HzkCXHstpdN97rlgQ7EsUx6Zv/7r+Pjck0hD\nj0rnngLfeKURV4avwK1Sf8mKjNbeVphTzFiXu24adU/f/+7lf3e4bnRi9/ng+8+RD/5vdm2MW5e/\n9x4NZf99XL5MQ379+vBD//0re3HlzBHIKq01IKvDuCwcAYr3YtuiXRHFE4tTNBl4XXOPFuE098n8\n4V5tOpyAduFC8OTtdNJ9qs1Gk3gonDlD2+Phc08iDT0qnXsK/IXBC5A9gf3lVtxoH2zHTctvmuY+\np+d/9/KT+eDj2eUXLgQm+fTu4+LFyIb+5csNkFVH4PdVBy6jETbbrojiicUpmgy87nOPFuF87pP5\nwyM1rRYVBSfntlhIoLTbKTtiKBQXx8/nniAfejR+83jxRRlFMBsC+8soGrHUunL0UDkURZ3iPlXY\n7Tywnij8715+Mh98PLu8qIiG8vh9FBZGNvRzWRXMQbnxU5GLyojjicUpmgy8rrlHi3CauyzT/72C\nntVKuVllOXKzb2UlWQ38hcdNm4g/cIAm8fGGYrOZ+AMH4uNzn8UaerR85aJKLEpbFKi5p6+F3LUC\nR7t7oCgqqpYVIm/54ijrXozGvmEcOH8Roij41ZMesf/dy9+xZT1ee+UY5E6fD37ecvLBx7PLKytp\nKPtr7rm5xEcy9JmwE4vUTQGae666CTizEwdaI4snFqdoMvCJ1NyZN5NevLFhwwZeW1s7tS8PDgKH\nDtHbGf4C15o1QFMT8d5H8c3NwI030qP49PSJy4fizWayDLS10VOte++l+rzlGaO0ex0dZCnwd8tE\nUn8k/NDQxD73WNTvcGCweg0+6G9Crpo6ppVfFhxYvWgNTlwJzTddaUKqMTVAn16TYN5sMOODjg/Q\nNtiGHEMBqlxfw6JcwKVKMAlWOPrSsaZ6CE39hyKvO3MrmmrTkZo1FFjPGjalpk8xkVum5ZyElUXk\nlmltEePd5SgvB154gSbXggJyy8hy5EPfYFTwftdeOGyNSLVX4ro8n1sm0nhicYpqnd+6NfayDGOs\njnO+IVy55Lxyj9TnftNN9PM5VbNvZibwqU8R39NDPyrjy3/hC776s7N9McbL5z5LNfSp8JmWTHxq\nJfVX68UhKFc8sBmzYRtdhEMWgT67FF3dXm3dmBFYzxSbPnueiH/47MaxITVsT0iXY3CQVgmayI8f\nfuiL2OzahUWLduHKFSAzfWpxTvcU1Tqv+9yjRSQ+91jw4cy+0zUT2+20j+9/n8y+GzcCn/88XTbF\n0YwbLx96PHmD0Q0wc3AT2Kzo7Fei9LlrZ0gRP5HvnvgBeQjff/MdnDzFsXGdEZ+vvhNm2R2VT7/7\nqoLvPnsOp0+lYEXJCJ777nIoijjjp9xs43XNPVqE09xjwYcz+4arJ5yZuLGRluz7znd8Gf737AHe\nfJNWWbBY4iYMRpMnPVn4tQurwARLcBNk2dDYp8xIbpmZHlLEczS2hfLdL0Nj23m8e+xjfOfJEfDO\nTwNuK/b8QcKbSz7EP9+fAovZFJFP/8MjCu76vANcJtPA5VPA6nfs+OULqWN+/ERr2cnC6z73aBGP\n3DLhzL7TNROLIq2oO/6Zh6oS/8ADcTPjRpMnPVn47l4XoDqxbVtqYBP02Wcwt8zMDimKfyi0795h\nhLjgNGraz4F33gu4R2cVtw3qpfWoaX8BD3z2uoh8+j/49bnRiZ2NDUuPZMMPfn0Gj35tpSb848nC\n6z73aBGP3DLhzL7TNROLIi2+HQqtrXFNvh1NnvRk4bvcHJzLsNlSA5tgqpq7BoaUL/4QvvtRP37r\nqRTAPU4LcFvR2pKCnIUeX/lJjutMS0rwmARwpjVFM1p2svC6zz1axCOfeziz73TNxIoClJSEPr6S\nkriacbXoW58ubzC6kTKquQc0gc2KkL71GPrZZ2pI+eKfuA1KVkuAUQrciVFCySop4jz1xaUjIYdl\nccmIZvzjycLrmnu0iIfmHs7sG66ecGbixkbgT/6E5Bt/aUYQiI+j2Xduae7paDxXGMK3viRmfvaZ\nGlKRPDP4k20DeO/Fo+CXqukK3ihBWFKLP9k2gKNdRxHJs4T/c/dyHHrbDi77BGOD1Y7/c/dyzWjZ\nycLrmnu0iIfmbjRS/phQZt9I6/nJT0Kbib3ld+yg1ZS+/32qt7qa3DI5OXEVBueU5t7FIPaVYduy\nxQG+9Ul5DQ2pcM8MdhTfiOrX7fj+r59Dc5MR1euM+Pyfz0eO7caInyWkWUW8+VoqvvvsGZxuScGK\nleSWOXtG1IyWnSy8rrlHC0miK1w2+sCHMfrc1xdb3mCgS6tt22ifg4PR1TM4SFfhVitN6gZDcHmL\nBfj2t6n+3Fx66DrTxzWOl+x9EAQBbJRnjEEQSMdNVn7ElQKoMhhLDdEEDMydAcgZYOZxTcMFQDGA\nMcGP52DGYUCQwQxmCM409PWxhAypPrsEQRDBRmyAbBuNXw5oA4vJiG//9XpIIxJy03JxefhycJsN\nyRCEjAn3azWL+OZ9K2E2U5zOOJ1ys43Xfe7RwmKhp1MNDfSTmZdH90Dl5eQ0kWVq3RMnaFWi228n\nzu323TOZzYHlZ4JfsQL47/+mZGLLlgGf+ASQlhZdnEYjTfivvQYsXAisXk0yTgzjtFSVo629BrIi\nQxREKKoCs2hGeU45ajpqIHkktPa2omOwA0WZRbh/8/2o6Z+4fCj+UNsh7DmzBxcGLqAoswi3Ft8a\nUJ4xhhPdJ/Dx8Me4vfR2yCMy3Ko74vprOmow7HLif99NwfnmDBSucOITCzegpyPSpuQ41HQBe96u\nx4ULBhQVeXDrhnUoLy9ETXMHZKljTK4xu/JRXr4UNTVsRoaOIABPP01ZpJcuBe65h7aZzUB5lRWH\nT4ygoe19dLlPI8+4ElUFK1C+ft5Y23BVwNtvM7S1ZOHmawwo3ngRHuYKaLOSvM147g06jfLz6TGP\n1RrbU+LwYe2foiUldDc1WTtMJ87NmxM3TSbn5K4owH/9F/WId+3QgweBm2+m7aoKPPMM2RDcblr1\nID8fuO8+ankvJkq9EAve4yGZZXCQPp85A7z/PvDEE77y4eJUVTrOjg66BDAa6enb3/xNbONHaJ6D\nQ+UqflT3I7QNtsGtuGESTWj8uBH3rL4nZPlQcHlc+Mb+b0Dy0IO+M31ncKj90Fi2RpWreKb2GVwc\nvAi34sb+i/uRn56P+9bdBxFi2Po5ODwejm//1SY4L64B3FacMUo4tOQ0Hv3/KiGKYtimdAn9+MZj\nPZDabvZ9v6AZNx1iQGYHcHURIAJQAOS0gRszAQRfkk23S9xu4JvfpK4HKA3BsWP0OoTZDHikVPxX\n4+O44DwGt+qCUTDhYP9a3Fz+BgCMtsOGsXZofV2CtTAVj/z4CEQj1akqDH95VyrqjpKEYDSSavj3\nfx+7IeXxaP8UVVXgL/+S3j+YqB2mG2cikZyT+5tvkmjpNQy73fT5zTcpcVdjI+UO9W6XJNructHS\nNt57zYEB6k2vbBJL/plnfBO7F04nXc5s3Urlw8VZX0+f/b3y3lypZWUxi9/pGEDBvAJYU6yQPTLM\nBjOkEQkD8gC6nd24NHQJboVidCkuNFxpwI7lO3BD4Q1B5UPV8+SHT45N7F5IHuL/rOzP0HilEe1D\n7WP7kEYktA22waW4sGrBqrD1D8gDOHwgY3RC8/m7nZdK0HS1DndfvzFsUz75ywZIbRsDvi+1rcKT\nP/8D/uw2I6zLHJCdAswWFZIoYWBYRkFBRsyHzhNP+CZ2L1SV1jv/x38E3jz9NtqcJ+FW6UDcqow2\n50m8eeYtFOcV4MVXhoPaQbpYgabDnbj7jnSYDWa8904qGurFsZS7bjc9Furujt0pkQyn6Hvv0Z3F\nZO0w3TidTkrlkAhEZIVkjH2SMdbKGDvLGPvnScrdwRjjjLGwSW2mhdZWX4944XJRL6gqXSWH2n72\nLP2fcyqXlUV/vT+7seRPngwde3Ozr3y4OCfbPsU4uarCbuDosQB2AwdXVVhtWVBVFd4kcpxzqCrZ\nA8/2ng3Kky57ZJzrOxeyfKh6Tl4N3RbNV5uhqirO9J2ByxN4nC6PC2f7zkZUf5Y1CyePGUP4uy04\ncVqapCk5zp53QVU5TjbzkN9vPpECVVXATYOA7Qq4aRCqSlZCVeXgBjtg6QE32KGqfNpD59SpkE2F\nU6eoXJu7Aa4QudTbpdNQVRVNxwyh2+G4YazNzp60QXaygCKyTJkoY3VKeCfAwPbW1il69mzwewfj\n22G6cWraCskYEwH8AMAOAJcAHGWMvc45bx5XzgbgawCOzESgAdi0iR5H+/eMyUSv+ikK8Skpvp9b\n73anM34Ww7Iy4Pjx4NhLSwPznk4WpywHH2dKCm1XlJil9p0shW/RvCKkCCljKxwBgDXFisLMwoit\nijcU3IDDHYeDmuL6guuhcAXyiBy0D5PBBOeIM9C+N4kVsmyNE8dflXxXrABgdCLNZJ24KY0qnKY2\nKOkCbtiShsMvBX//+k3zoPDuGV1+z5/PyqLnAuPh9VFfV1yJHx63QFZ8d0Im0YJrC6uhcAW2grPk\ncx/fDkvP4miXC4qioKj0k0hNpdWP/IdVYWHsTonrrgN++ENtn6JFRQjbDtONU+tWyI0AznLOzwMA\nY+xFALcBaB5X7jEA3wXwTzGNMBS2bqUnH94reJOJPldU0JXx7t10qdPSQtstFroX3L2bxMB4+KAq\nKoC33w6UZmw24LOf9ZUPF+fmzXQ8Z87QaDKZ6Mdh9+5AS2UMlsebyF5XnVuN1QtWo/lqM1weF0wG\nEyoXVaI6txrp5vSIrIr3Vt6Lp448heER31mUlpKGeyvvRevVVuyu2o1TV0+hpbcFLo8LlhQLirOK\nsbtqNzzcE5EV8rO3ZeCtXzTDfmHVWK701KWtePBL68EVv6Y8y+GUAJNFRelqCbv/1gE55TLuvbMM\nTz3VjGG/76cVncS9dy5Ea2/vjC6/58+vXw/ceitNDl6IIvDv/05XguXzrkFJTjFae0/D5ZFhMphR\nkl2MikUVaO5uxoNfXIf6V4OP48EvrgMXqC2HFl7F2ioP6o6mjMVSXk6PiGJ1SpSXa/8UHRqidMpe\nzT1UO0w3Tq1bIRcD6PD7fAnAJv8CjLF1API5528yxiac3Blj9wG4DwCWLl0afbReuFzA44/TpNfa\nSqOmuNiXlzQzkzRv7+P69etpJQB/8Sse7x6/8w7wn/9J93Br1wJf+hLF6J9XNVycjz9O94l1dXQp\nsGULfdc/P2sMUvtO9Eq+McWIpz/1NA53HEZrbysWpS3CruJd6HH2RPwK/7B7GL/6819hz+k9OPbx\nMaxduBa3rrwVw+5heFP1PrPrGRzuOIyGKw1Yn7sey7OWI9Pia4dw6QcsZiP27WN4/uX9OHaMYXGB\njNsrrkf2PN9Tr8cfB8519aOu+SqqKgVsuXGQmlISMDwyiF+9ehV73qTvr13LceunDBgeMU24z1gt\nvzeef/ll4Ec/ohu/igrgW9+iSaWnBxh0uPD4jY/jTO8ZtPa2oiS7BMXZxRiUByGKIrKtmXhnn4Ln\nX96P2noPqtelYNenRGSnLQpoy6ee7cMb/7MQV67Q6bNlC8USy5TCyXCKPvUU8MYbmLAdphtnUlsh\nGWMCgP8AcG+4spzzZwE8C9BiHVPeqVfIqqwk+cP7Mzk+72ZlJY0obyb9eOf7XLSIfGzeDP4uV+jy\nk8UJkP9q+XKqp79/yvFMNbWvNCKhbH4ZKhdVwuF2YH7afFwZvhJ1PfesuQefLv/02NXv+PKViypR\nnFU8tnBGtPW7RAmfvt2Me+4wobvXBHRYYZftY6lugTTcfpuI5dedQaoxFf2uEDHekYFP3+6aMMZ4\npAj2vuzkXfzB4aCrREXxpiwGKhdVomx+2YRx3nNHBm6/1TFhW86fZ8aGDRypmZQ6WBqJLBVwtLzW\nT9H584ENG3ynqCRFf4pOVr+mNXcAnQDy/T4vGeW8sAFYDeCPoy9KLALwOmPsTznnU1xqKQwiST/A\nGN1LMQZ85jN0Txt5XtVZyScyzUCdUodf1f0KH0sfY6F1Ie4svzOgPBMYWnpawBjDZ9TPQOUqai/W\nTnm/axdWgmVdxoGTA2OaeEVpOhp6W/BS80vgnKN0fim4yiOOURRFjHhGoEJFbVctPNwDAQJSDClj\nsSzMtOG5GnoLdfFier1h/Xpfl7x3QMF5YS861QZcs6wKn1i0E42NYlTpB+quusPGGT6V8cw8M4gl\nX1FBV8svvUQPKEtL6a8GT62k1dyPAihmjBWBJvXPAbjbu5FzPgggx/uZMfZHAP84YxM7ED79wNat\n5BtrbaV7o9deo5/d730vfoKeBvlEpRnosHfgkfcfwene02MWxuarzajMrYTIRGxduhX/+O4/orW3\nFbJHxmutr6E4qxjf2/G9iDX3oPQDw91AViO2XbsCLlmAwTiC+w9+Fmf6zsA54oTZYEbJ1RI8ueNJ\nyB45bIzbCrdBGpHwrfe+FfD8oTynHI/d8BisKVYMOR3YvkNB3VEhQMNdv566BIKCNzJvRvPQEbjg\nwIcXU/HeC5vwQN4+bNsmRtS1HVcHwsYZWV/NzDODWPEGA3D//b7HTd4Xjp58ckqPm/T0A6HAOfcw\nxr4MYB/oNY7nOecnGWOPAqjlnL8+00EGIVzK36Ymmti9KXcliT43NdHj+vHltZIfdJam9t3Tumds\n4gYAp8eJ1t5W7Gndg5U5K9HU3YTW3lY4PdRf0gi9EdvU3YRrC66d0n67hrvAOYctgzTxg22HQsbQ\n1N2E0vmlYWO0mWxovNKIU1dPjZWRPTJOXT2F072ncW3BtTjwrg31deKYQ0SW6WFcQwO9adpg34tT\n9iOQVXqwLKvDqO8+ggbbXuzK3RVR1+5pfSNsnJH31cw8M4gFf/Cg79oMoFPZewqXlmru1NKk5h6R\nz51z/hbnfCXnfDnn/PFR7tuhJnbO+fUzetUOhE/5e/x4sIHV5aKRoYU8oAniE5Xat22oLaSPvX2o\nHYqi4Hj38SAvvcvjQlN309RT/sKAFCFljDvefTwoBtkjo6m7KaIY7S476q/Uh/T8N3S0wm7nODOB\nf/zsWeqSM8MNkJVAj7qsOHB2uAF22Y6rjh7YZTsUhY92IQ/i2xytYeOcVh/GMMXxdPjjx4P95bJM\np7AGTy095W/MEE5z14KJVoN8ojT36/Kvww+P/jBgYjQZTLg2/9qY+dyDNfe1YGBjnDwiw2QwBcSQ\nIqTAOeKEwpWwMR64eACc86A4U5gJ6tAiHKjtRlH+gkl900XmKqQgFW74ClhTUlGYsxAHTraEXDZv\nvCZ+Xc5G/LDBNGmc0+rDGC4rOB0+xq946Jp70iCc5q4FE60G+URp7uULy1GSXYLW3tYxrboku2TM\nmx0Ln3tIzR0Y4zYv3ozmq81jmrvJYEJpdil2V+2G7JHDxritcBuqF1ejuafZp7kLqSjPXI+7SnfD\nKtowZEvH2rVsQt909bydWJ25iTR31QGTkIrKBetRXZqL9FJDyGXzxmvi5QsqwsY5rT6M4bKC0+Fj\n/IqHrrknDcJp7lox0WqMT5TmPigPTurNjoXPfULN3Y97/MbHca7vHOou16FqURW25G+BKIjokXrC\nxmgz2WAz2fD0p57G3rN70d3vxAJ5K3YW74TIRIrFOLlv2mgQ8fQ1+3C4ey9ahxqxiFdi14ZK9LAT\nsFlDL5s3XhOPKM7p9KFGNHcgpq94zEnNPTknd3/NfTKzaaJNtAnkeWcnhmU7ZMUFs2hCmjJ1n3us\n+HDe7On63P15AwyAgAAOHLi97HYsn7ecfO5yf9QxmkQTNuRuwJqyrWiqzYA0HJ1vWhoWUSbuQuWi\nXXA4gPkZg7jST8v7uVwMJhOHAnXabTnlvso0o1PhsMvDY+8HzIT/PRIeiNkrHgnjdc09WkTic5/D\n/FRyyMy4z93TiF/9rgsfn8vDwuVduPO2tIDynHG8d+E99Mv9+NOVf4rKRZWo7Qr0uS805+G5P1zG\nB0clLF7Zj2UbTmP9ksqINPdIYvzI8xGeOPwEOoY6kJ+ej69s/Eqwz92tQj1zM2q7bfB4KPd6Soqv\nSxYu9K22FMrnvn8/cOgQJdbauBF4/oZ0KOOW/asoykeD2jChHz8WfbLQshDPdT6HD9o/wOL0xViW\nuQzr89Ynhf/dn3/vPeD8eZpYr7mGlkyI9X6n88qMrrlHi3gss5fE/FRyyMyoz32oC4/81WacPj4P\nslOE2aKgeU8/Kt+k8puXbMYdL98xlnumqbsJaSlp+N2dvwNnnPKhOB3YfvMI6o6uh8spwmRRUF45\nhPXPfRiR5h4uxouDF/HF339xLIYLAxdQf7kelX/j53N3u/CtL1WjuTEdLicb09Qfe4yu0IaGgO3b\ng3OVeH3uikKyjd1Ow/jCBeDttxl++lPf8n4GZsL9NXfijPMInB4pyI8fiz4Zkoew/ZfbUXe5LsCv\nvz53veb97/48QBJYczNt+/BDmuwfeCB2+53uKzOJ1NwjskJqDv6ae04O/RXFQOFrDvNjOWQybPAs\nzIGaYYMgigE6a441BzaTDWIc+D1veNB6fB6ckgGcMzglA1qPz8OeNzwQRREvn3o5IKkYAAyPDOPl\nUy+P1dNwaBHqa42QR+uQJQNONWag4dCikPv1wIMRPhJxjE988ETIGJ744Imx8qdrluPUsQxfDKM+\n9tOnqekbGigFvyzTm5T+Pve+PsoZ453YvRgcBF5+mcFmzECOORdNA4fR6vgIkscBDh7gx49VnzR8\n3ID6y2Tr5OBjfv2Gjxt8Wn+GipyFHtgyVIii1/+umSGOvj5q11OnAtu7vp74WO3X/5UZzgNfmYmk\nHilwGYO4Ijkn93A+9znEc0WBXbajx3EVdtkOriTOzz6hz701Cy6nGNCFLqeI9tNUvu5yXchurr9c\nP1bPmZNpkKXA4So7BZxtThvdL+nW1DwcBqQE+NzDxThZznlv3fUfWYJjkGkysdvJ2REqP7jX5z5Z\niv8xf3dPcL522SOj6XIrFEWNSZ+c6T0T0q9/tu+s5v3v/vxk7R2r/U73lRldc48WuuauWW09pM99\now0/tCiQJd9wM1kUXFttoxzkxtDCZJoxbcznHjIHuVFB4RIL8lKXoLFvOEC3XltYBjbvXGxyzveR\nJs5TGVKMC+F2+Y4jJYU02AMHwucHnyzF/5i/W6mCSUgde4sVIC+98+pCKH2FyFu+eNp9UjSvCKkp\nqQF3KilCCgozCzXvf/fni4qCX2exWqm9Y7Xf2Z7PXXvQNXdNausT+tyv7URJhQWtx+eN6eUlFf2o\nuNaJ5h4RD259EPUv1wfle39w64M+zX0sB7mBmsDMUV7OUb0yH11dDGKfT7c2CVZ0d3PA0hZxjBPm\nnC/9Globc7Bt2WJU50to/gNH80mvs4U09bvu8mnuk+UH/+xngbfeCpRmMjKInzdv1N9t3Inmhk2j\nmrsDJtGK0vQq7F79Vch989DVNxQTzX3torVBmnt1brXm/e/+fHU1LXLu1dxNJjLIVVcD6emx2e9s\nz+euPYTzuc8Rfir52RPic3f14fEXruBMzQq0nrSgZJUTxX9yFoOuRaAc5Nl45y/ewfONz6O2qxbV\nedXYtXIXslOzx7o8OAc5w5Ytgl/ubdKtbaOLVnexy+CeyGOcMOe8pPrqNmbg6R8Ce/fSOpsLFgA7\nd1LXeLtqMp+7xUJroT7/PC16vWIFLcbc1+fftSIeL9uHc2wv6jobUZVbiS0LyEvfIwJ9dikmffLU\nzU/hjTNv4MrwFZRkl2BL/hb0yX1J4X/38kYj8PTT9DpLaytl2N61i3zwsdrvnM7nnhBE6nPXMB/K\nh86i5BPtW4/a537dJZRd470aDC5/z5p7cHvJ7WFykE/kIQ/0Zhu4BUhRpp9zPj3Y971hQxrWrGFo\naqIYovG5u1zApz9Naf4dDpqQenrG+btVEbev2YXlyi6kWoH+Xr+hY7Oicwq++KC2TJuPDbkbkGpM\nhUtxQRqRJm+fGOSpnwlekkjuqqyk9pw/n35YY73f2ZrPXXtIcs19Iq18WV4ezkfBR6qth8tBroV8\n7pHUk24x4YEfvofzpzKwYOkg7r72GmyuNiIvj6OxLdCbvbYkA8zkCaijYmEFGrqi9I9H6fsO53Of\nti87Kx2N43zxVctio8UnQ86ZZON1zT1aJLnmPplWHi0fTmcNl4NcC/ncI6nn44F+zFskAc7rAQDt\nF4B/OzKAF38JdPU5g7zZ3Z5jgMLH6jAwA+5/9/4J87nHIu95OJ87EAtfdvDzBUdfeky0+GTIOZNs\nvO5zjxaSRK8GstH0qozR576+hPFcEGA3MvRYALuRgU/CS/Y+MEHAkIXhig0YsjAwgfKJRMsLgoDR\nFbDAGIMwjq/tqh3LQe7vaa7tqg1ZfqJ6psN7c6U7Pc4A7/ae1j0R1/PY/68CznkAmO+fnInHfvoB\n+uwSBIGBWYYA2xUwyxBGuBsj6shYHR91fYTW3lZII9JYDC29Lfio66PIjklUAdEDJqrED8lBQ6G+\nPrTPvb6ehk59fWhfdn19tEOQgbkzgOFcMHcGBIGNtoEINmIDpBywERsEQYxd34Y43gSfcknBJ9Ln\nnitGfz4AAB2lSURBVJxX7hYLvb8ty757ILOZLpNqauLO8/JytDfXoEOS6QGnoiDfZUZ+eTk6QvDp\nqeVoa6+BrMgQBRGKqsAsmlGeU46ajtjyRzqPjC2C4YXT48RHXR8h3ZQ+Y/v159sG20L6qtsH22EU\njBHVc/5YwJrsY7jQasO8VDNq2tsCyquKCjCgR+qBKIg42HYwZAwH2w9inmne5MfU3AW5l0MUGRSF\nw5ytorzcjJpxQ9A7sQe0tZP4L36R/joDu2Jsgs/MnObQrLKi5qQaHOe6ebHp2/zNQcebwFMuafjN\nm6c2xcUCyXnlPhE4Twg/ZOJoywQWOIAcB/1tywQupYfmh0yh6+GIPb8kfQmMojFgu1E0YrFt8Yzu\n1x/l88spcZcfTKIJZfPLIq4nb9WFkGVzSzrATUNh61mcvjgoBv92mPCY5HSgvxBI7QasPfS3vxDc\nlR5UNj+frjsC9mEknnP6awzsClgsxAftN9qhGUWcwFT6Nsp4dD7hSM4rd6cTKCigR9GyTD+RkgQM\nDCSElxwDkIoL4FhmheCUoVrMkEQJfXJovl8eQMG8AlhTrGMatDQiYWAG+JXZK1HbVRukud9WchtS\njakztl9/foF1AUpzSgM095XZK1G5sBLdUndE9Xzn4QF8Zl//qDQzCnM/vvO4iAG5P6j8paFL4Jwj\nPyMfskfGqvmrcLTraEAMJdkluGv1XXApromPadiNgtQyWOcthqxIMItWSP3pGBhgQUNhxQpKKOVd\nRcirue/cSUNn5066uvP3Za9eTbzNNt2hyULH6bgSm74ddmnplEsa3ukMtE3GE8k5uVut9Fqg9+eS\nc/rs9SPNIM87OzFs4JAtgFnkSFPVMUvioInDZQBMIofi9tnTJuL5aP2cc6jqzPCiIOKJTzyB99ve\nh8PjQKohFdcVXAcP98zofv35+Wnz8ei2R3Hy6km0XG1BaU4pVuWswvy0+bgyfCWieqxmA06e68Od\nX3kP7SeXoHB1J77zcBqYQQxZPoWlAALGOIEJeHTbo+iVe/GHC39A+fxyrF24FgITJj+mTDM6VQYu\nZwCuDHDTxEOEMeDFF4EXXgAuXaKT/Zpr6GT3ln/iCXLStLUBS5YA995Lk31shmyIOG1WdPZzcFcq\n4EoDN3GocETft5lmdCbmlEtqXrdCRosEWSGT5XX/UPz6vPW4ZcUteOvsWzjUfij+8bBGuD1uFGcX\nw+1xAwxR17NyfiG+86/NaLj8NkRRRF135Mvseev4yzV/iWxLNhouN6D+Sn3MLYD5+eSOEQT6d+hQ\nYPlDh6j80qW+8k1NMziUY2Wd1K2QuhUyLkiQFTJZXvdPBD8kD+EXTb/A/vP7sWrBKqxduDYqq6U0\nIuHYx8fw0smX0OvsRbY5O6Slsq6rDh32DhRmFGLr0q0T2hijTfnb5egCVBHGc3+OUyfMKFstAwvb\nZ4EFMEbWyaRvh6nzkkRvFL/0EtDbC2RnJ4cVMjkn9wSlH0iW1/3jzSuqgq/v+zpae1vHdNrVC1bj\nv3f+d0T1WFOseOB/H8CJ7hOQPTJeaXkFJdkleGbXM7CZqLxjxIHP/fZzON59HC6Pa2wfD217KOJl\n9iaLpcfeh4f+ahNaj8+DLAkwW1WUVOTix89r77X76PnA1AyyOIU0BhpMPxAP3mql9xBOnCD+lVco\nrcQzz0RWTyLTDySnWyYOKX+TIZWuVvj9F/ajpbclwMfe1N2E/Rf2R1TP/gv70dTdFPD9lt4W7L+w\nf6z8R5c+womeE2N+fe8+Prr0Ucj6DTBElfL3+Ad5aDmWCadDpJzzDhEtxzJx/GjaTA+1xPA2K8an\nSZ40pbAGU/7Gg9+/n2Qzbz53p5OSiO3fr6f8nRnMsOaezNp6Ivg/nP8DXB5XQBfJHhnvnnsXq+av\nClvPu+feDfKguzwuvHvuXShcgaIoONt/Fs4RZ9A+zvWfi8kye+0tOXA5WWAMThEdF6zIztCOhpsw\nLX6Oau7vvhv87oLLRbyi6Jp77DHDmruurUfHi0zEh50fBkzQZoMZO5bvgMjEsPXsWL4j6Psmgwk7\nlu/Autx1MIkmqFyFJcUCaUQK2MfyectjorkvXeaEyWQNOJFNJiA/nyVc89WEFj9HNfcdOyhNxPhx\nsWMHsG6drrnHHjOsuevaenT89qLteOnkS0Ga+/ai7ZOnkR3ltxdtx6str45p7l4P+vai7RAFEQCw\ncclGrJ6/Okhz37hkY0w094rqYZSWpo6tlWk2U1rhiorYppDVFh+FFj9HNfft24FXX/Vp7jQuiBfF\n8PXoKX+jxQyn/E22VLpa4B/Z9kiQhzxsGtlRXhqR8PC2h3Hs42No7mnGjUU3ItucDWlEGisPDrz4\n6RfxQsMLuGS/hIKMAlyTfw1kjzxaT1dAGlwDUgCBR54Cd54ZjzzCcbLFiZbTHKUrGVaVWjB/PpuR\nFLKa5W1WdPaHSJWs0ZS/8Ugp/PDD5JZpbgZuvJHcMuNTPespf2OFGdbcbbq2PiU+ag95CP7O8jvH\n/Pjjy+fb8rE+bz2EywIEQcCh9kOj9Sye9jJ7edlpaJx3Du7MfhRvYXB7ODBv4tS+idaCZ06Lt6Gx\nT9E19xD8nXcCt9xCq2npPveZwgxr7va++GnWktuFYwfz8NJrRvRu6UP2mujqAQeMBiNOXT2Fspwy\ngEOTPvdwcU6rHXqHp73MXjSpfScbUhfbFPzm/Tq0npdQtsKKL9y0HiMuMWl82V19dl1znyUpfyOa\n3BljnwTwFAARwI855/82bvs/APgrAB4APQC+xDlvi3GsPsRBc4+HZm012PDA59fhREMqZEnAKz9R\nUVKxCc/8T9uYv3tSb/ZwDx468FCA1l2SXYIf3/rjuGru4Xzu4eKcbjuM6cHTWGaPeAE2mwpbhkq8\nJESlNQ8NK1h3yzHYL5YDbgsajE7s+f0x/PJ7a9HXJ8ZFI56uL1vX3GPLa9rnzhgTAfwAwE4A5QDu\nYoyVjyvWAGAD57wCwG8BPBHrQAMQQ597Iv3s+/cZ0VRvDfJW799njMyb3X08yF/e0tuC493H4xJ/\npD73cHFOtx3Ig81hl+246uiBXbbDwC1ISZko9pnxd//mj3WwX1wJuNMAiIA7DfaLK/GbP9YljS97\nQv/7HPW5T5fXuua+EcBZzvl5AGCMvQjgNgDN3gKc8/f8ytcA+ItYBhmEGGnuifazv3v4KmRn4O+r\nyyni3cM9UFYeDVtP+0B7kL/c5XGhY7AD2ZZszfjcw8U53XYItRReqGX2JtLnY+XvPnlGAtzjcv66\nLWg+IyWNL3ti/3v6nNfcZ2NumcUAOvw+XwIQeuUEwm4Ae6cTVFjESHNPtJ99x5YcfPg/KmTJN7GZ\nLAp2bJmPdXnV4b3ZmUthMpiC/OH5Gfma8rmHi3O67RBKLx+/zN5k+nys/N0tdiuajM7RK/dRGJ0o\nL7YmjS97Qv9712z1+89xzT1SMMb+AsAGANsm2H4fgPsAYOnSpVPfUYw090T72bff7Mar6yScqE+F\n7BRgtqgoWTuA7Te7IYo5YeupWFCB0uzSIC27YkEFepw9mvG5h4tzuu0QSi/v6h0J7XMPoc/Hyt99\n1/Xr8eaeY6PSjAUwOmErPI27rl+fNL7sCf3vGtOyk4XXus+9E0C+3+clo1wAGGOfAPAggG2cc9f4\n7QDAOX8WwLMAsGHDBh51tF5MwefOOzsxLNshKy6YRRPSlMT72SWPHQ//+EMcO5iH5hNG3Lh5HrLX\nHIXkSYXLFYE3O20+Htn2yIR50rXicw8X53TbIRRvgAEQWID3XcFoXnKFwy4Pw6XIMIlmKEpaGH93\niPJZDJ2dgbzI0lD/1lp86+lanL5oR+lKA75wcyVGhgWN+rIjOy4frz0futZ5rWvuRwEUM8aKQJP6\n5wDc7V+AMVYF4BkAn+Scd8c8yvGIUnNPtLYell+u4M6tVaP+7hAe4xnOkx4Xn3skcU6nHYJyy1SC\n9S/HgYvt47RjW5A+X1U2D3lZRRP4u4P1/KqyecjLW4bGtvNBfGHBMtx1WzYaTgkQDQwftp4ZLR9/\nv/zkvmyOxrbIj2vO+f3ngubOOfcwxr4MYB/ICvk85/wkY+xRALWc89cB/DuANAAvj66a3s45/9MZ\nizpKzT3R2vp085hrJYfMZHy/1I/7370ff7z4R1QuqsSnyz4Nm8mWsHi6e11AdyG2LcsP1tZD+dkd\nxuj87w5jdHyUfvkZ5ydrh2SIP0l4zWvunPO3ALw1jvu23/8/EeO4JkeUmnuitfXp5DHXUg6ZiXi3\nx43P//7zGB4ZBgAc7z6OV1texb579kEWExNnl5uDcxdsxuwQ2noIP3u0/vdoea35xCdthySIP0l4\nTfvcNYlJfO7JlIc9kjzmWogzHP+juh+NTexe2N12/KjuRwmL02B0I4WZJ/BxJ6DNQnjxFYWP+qNj\nxUfhy462HSaNXzu+cq3xWtfctYcJNHfNa+vj+EjymGshznD8wbaDIbvpYNtB3FH2/9o7+9g46jOP\nf57ZtXe9GzsOdt4IzprUeXPj5BzSkgjaFHI90VwFvfbo5XrRlR46Snu6q3qnSkXQ49RQIf6406kS\nUpuDcr2KAwqtkCtaUA8oiAjTpCSExCglL3ZsQl797vHauzvP/TFL4jh2vGvv7szu/j5SlN3v/jTz\nPL9ZPx5/55nffMmTODcsbkWsqszXTsl3jFl699nrWXri2c7DtPEbL75oPXdfMo3n7mdvfbbrmPsh\nzpn09y+8z4mDJ644TJ+KfSqj9dzz5rk7o2zdGs1s7ZS8x5gj7z5XnnjW85CbtXfKTffScy9OW8a2\nUctiqFI4VwVDlYJaFvZQL2JZDFYJp6thsEoQy/URLcsifbEXEcHygX7jshtZU7eGqmAVglAVrGJ1\n3WpuXHajr+KcSd/ZspN5FRNu3AHmVcxjZ8tOz+JMjFWQcOJI5RBUnUMqh7AspXfI9nDOBKkahOrT\nSNUgliW50wfjWJZemW8vU+uzmods9wvpzSDivi833b70bJmCU5Rn7lpVxcm+LrrtuHuhNJWiYSxM\nTbSZrpPtxFNxAlaAlJMiHAjTXN9Me7c/9bv+5C4OnT3EmeEzfGHNF4gn4hw8c9B3cc6kP3zrw7zf\n9z4vHXuJ2PwYtzXdxuFzhz2Lx4lXw9BSznGBj1ohw2MNNEdraD/Z5Ys5y6nesJn2vpPE7e7L821u\noL2j+0o9V/Mw7X6X094uxOOXbIpwGJqbob2dstG3bPGuThZlcR8MQVctLDmP25yZgq56iIWmvi9K\n8a9uicXHF36cG5bcwK3X38qL77/oyzhn0oNWkG9v/jZr69ZeVgA8izM0iC7ohIGPXfyOUN+FhmKF\nj6UQengQarvg/JLL862RqfVczcN0+62sBa70I3SaWxdLVfeSoizudnIUe2WMkRURrNE4TlUYO2DT\nF+8ntiBGpCJy8TZ3O2HTP5UeH8L5VRu3vr2XsZZmTt28gXAoOv14j/ShsSHajrSx99Tei3d+RitN\nnDPpPYM9aHUvDaElF5c0sAM2/fG+vMZywb7Aafs03QPdxObHuHn5zYylxgowB33EVtpEVoxMyrd3\nGj1X8zDNfofjxGLziUQuLYNg29DfD7EYZaOPjkJtrTd1sig990hFhJTjMBBSTlfDQEhJOe6yrY7j\noOlfo6qKM5WeTLLpaw/Q+M37WPnDJ1n7T7vY9LX70WRy6vHTbSfPejKV5IFXH+C+V+7jyXefZNfr\nu7j/lftJpkycM+kVUkFloAINDUD1aTQ0gOOk8rrPRDLBt176Fo/seYQnDjzBw3se5ju//Q6JZKJA\nc5CaJt/p9DzutyaM41w6o1UFx3FbBstJN62QWVJdWU1KZ9/OtuKNDm4+eATLdrtUgvYo8w50cOrn\njzPvizt802LYcbbj4mJbAKPJUTrOdfD4/sfZsc7EefVWyA0IUtB9JjXJO6ffuTgP8WScQ2cPsffD\nvWxfud0Xx6pwrZ/msXymFXIWDI3PrZ1t1fnjBEYvX9usMp5gy4UqOjxq3ZtKP953/Ip10BNOgqqK\nKs9aDIslzrPD7hJHhdznY28/xkhi5Ip56Ozv9M2xKphuHsvneStkURZ3OzHN8qyZPhZuw3pSVSGC\n9qX+cqcqTGp9C4FAwDe39a9ftP6KddDDwTAti0ycMy4/MHxq6iV/87jPlXUrCQfDjCZHL5uHpmua\nfHOsCqab5QrM8gOzIVIxt1vIe25qoW/9apxoBBUhGQnTv2E1PTe1ZLWdfOsti1pYXbeaSEUEQS6u\ng96yyMQ54/IDBKmwKgq6z9bFrWxcupFwMHxxHtbWr6V1catvjlVhl1vwzzIAZvmBImGunnsqlWLw\nsQf586MWR155lv2LHU5sWUOy+w3f+ZcPfvpBLLF4tuNZHHVYs3ANb5w0cfrRc29d2srLO1/moTce\nYs/JPSyrWcaK2hWeLsNsPHfjuRcVc/XcP9I/+EwLx9cEubYySp0ffMpp9JYlLQStINHKqC/iKYY4\nvfDcR8ZHODN6hs3LNrPt+m2ez4Hx3L3XjeeeJXP13I1e+vpUnvtgfISH/usAv/9Dgk/ckOTv7lxK\nZUUgu+s1TorXul7jhaMvsDi6mM81fa6olmcumG48d88996Is7hM990I9Rs7oxaW7j9njoiZOkL/5\ni3qGTzTDeIR3fmnz9GMd/KKtn5Rmtm07YfO9V79Hx/kOxpJjhIIh2o60seuWXRe/k36aA0/1qz6e\n0D+PwSv3x+z5jlx47kYvbX2y5/7b34QYPnEvjKdPq8arGT7RzA+e+BH/8tXVGW37aN9RDp09xLgz\nDlzqY3/q0FM0LWjy3Rx467mbJYKN5z4LcuW5G7109cme+y+7bBifdBo1HmH4ZFPGvfjH+o+RcBKX\nbSLhJLDEYmvjVt/Ngbe6WSLYeO6zwHjuRs/Wc79hYw/7f2FfOnMHqBxlY6tFIJBZL/7GJRt5ruO5\nK/rYW5e0Uh2q9t0ceK+bx/WZPvcsmWufu9FLX5/c537n7fOYd30HVA4DKagcZt71h7nz9nkZb3tV\n3SrW1q+9oo99Vd0qX86BL/Uy6383nnuWGM/d6Nl67qlUiieeXc+BN5Rfvd5NbWMnt3w2zpsfOFlt\n+/MrP8/2ldupsCpcSwaLt0+/7cs58KVeZv3vxnPPEuO5G322fe73fOVattx6gWjlJ+bc0//u6Xd9\n0dNfVHqZ9b8bzz1LjOdu9Gw9dz/GWJa68dwLhvHcjV6Suhdryxg9U89dGYoPcX7kHEPxIVIp9Z1X\nbjx3jzCeu9Fn47n7Lcay1Mus/9147lliPHejz9Zz91OM5amXV/+78dyzxHjuRjeeezHr5dP/bjz3\nLDGeu9GN517MusPQkKa9aSWVckq2/9147lliPHejG8+9WPVlHOgd5rXjnQQCFqmUQ+uKRq79WE1J\n9r8bzz1LjOdudOO5F6l+YZhA71q2rljGmGMTsiKM9NZw6pT4yisvBc89I1tGRG4TkSMiclREvjvF\n5yEReSb9+Vsi0pjrQCdiJ2wsy0JEPto/lmXRa/eWnK4oz7/3PE8ffpo3e94k5aR8Gaff9IQmSDgJ\nX8RSvnoASVSDXY8kqrGsAL2DcSwLRC1IBRG1sCzXs7YsSG8GEUpCt208Y8YzdxEJAI8CnwV6gL0i\n0qaqHROG3Q30qWqTiOwAHgH+Kh8BA1QFq+jq6yKeihOwAqScFOFAmOb6Ztq720tGT6QS7H57N92D\n3dgJm8pAJY3zG/n6DV8nEoz4Jk4/6k7KAYFz9jnPYylbveMU8QtKICCkUkq4zqG5OUR730nidvdF\nWyY81kBz83La24V4/JKtEQ5DczO0t1O0+pYt+aqCM5PJmfsngaOqelxVx4GngTsmjbkD+Gn69XPA\nNvnoV3kBUbSk9PfOv0dnfyd2wv31P54ap3Ogk/fOv+erOItJ91MsJa3Ha6CvEaJnIXLO/b+vERWg\ntgtGFsFIvft/bRdaOTj1dqbefNHoXpKJ574M6J7wvge4cboxqpoUkQGgDjg/cZCI3APcA7B8+fJZ\nhgyjyVFiC2JEKiLEk3HCwTB2wqY/3l9S+oHTBxhPjV+WeyKVwFGH2IKYb+L0o94z2IOq0jC/wfNY\nylIfHicWXUtkwTLiKZtwIILdV0P/yGliK20iK0aIj1qEqxzsgE3/cJxYbD6RiHv2Gw67lkZ/P8Ri\nFK0+Ogq1tbMudXOioK2QqrpbVTep6qaFCxfOejuRigiqSrQiSn2knmhFFFXlmsg1JaWvW7SOUDB0\nWe7hYJh1i9b5Kk4/6pVWJaFAyBexlKVeE0ZViAbnUx9eSjQ4H1XhmuoIqg7R6hT1i5NEq1OoOunx\nEI1Cfb37v6rbYljMut9bIT8AGia8vy6tTTWmR0SCwHzgQk4inIKaUA2NtY109ndiWRaO49BY28h1\n1dcxPDZcMnrTgiY2LN7A4bOHGUmMEAqGWF23mqYFTb6K049688JmALoGujyPpSz1+hqGG6Gz072w\n6DjQ2IirM8XP7nTjr4Ph4eLVa2ryVQVnRnQGsyhdrP8IbMMt4nuBr6jq4Qlj/gFoUdV70xdUv6iq\nX77adjdt2qT79u2bdeCqyuDYIHbCJlIRoSZU43aXlJgerYjy4rEX2f/hflbVreKmhpuoDlX7Lk4/\n6oBvYilPHQYHXXsiEnELncjVfnanG1/ceq4RkT+o6qYZx81U3NMb2w78JxAAfqKqPxCR7wP7VLVN\nRMLAz4BWoBfYoarHr7bNuRZ3g8FgKEcyLe4Z3cSkqr8Gfj1J+9cJr+PAndkGaTAYDIb8UJRryxgM\nBoPh6pjibjAYDCWIKe4Gg8FQgpjibjAYDCWIKe4Gg8FQgpjibjAYDCWIKe4Gg8FQgmR0E1Nedixy\nDujKwabqmbRAWYlj8i1dyilXMPnOlpiqzrg4l2fFPVeIyL5M7tYqFUy+pUs55Qom33xjbBmDwWAo\nQUxxNxgMhhKkFIr7bq8DKDAm39KlnHIFk29eKXrP3WAwGAxXUgpn7gaDwWCYRFEUdxG5TUSOiMhR\nEfnuFJ+HROSZ9OdviUhj4aPMHRnk+88i0iEiB0XkZRGJeRFnrpgp3wnjviQiKiJF3WGRSb4i8uX0\nMT4sIv9b6BhzSQbf5+Ui8qqI7E9/p7d7EWcuEJGfiMhZETk0zeciIj9Mz8VBEdmYt2BU1df/cB8Q\ncgxYAVQC7wDNk8Z8E/hR+vUO4Bmv485zvrcAkfTrb5R6vulx1cDrQDuwyeu483x8VwL7gQXp94u8\njjvP+e4GvpF+3Qx0eh33HPL9NLARODTN59uB3wACbAbeylcsxXDm/kngqKoeV9Vx4Gngjklj7gB+\nmn79HLBNJB8PuCoIM+arqq+qqp1+2477XNtiJZPjC7ALeASIFzK4PJBJvn8PPKqqfQCqerbAMeaS\nTPJV4KOnjc4HThUwvpyiqq/jPo1uOu4A/kdd2oFaEVmaj1iKobgvA7onvO9Ja1OOUdUkMADUFSS6\n3JNJvhO5G/dMoFiZMd/0n64NqvpCIQPLE5kc31XAKhHZIyLtInJbwaLLPZnk+2/AThHpwX3i2z8W\nJjRPyPbne9Zk9Jg9gz8RkZ3AJmCr17HkCxGxgP8A7vI4lEISxLVmPoP7V9nrItKiqv2eRpU//hr4\nb1X9dxHZAvxMRNapquN1YMVMMZy5fwA0THh/XVqbcoyIBHH/tLtQkOhyTyb5IiJ/CtwP3K6qYwWK\nLR/MlG81sA74nYh04vqUbUV8UTWT49sDtKlqQlVPAH/ELfbFSCb53g38HEBV3wTCuOuwlCIZ/Xzn\ngmIo7nuBlSJyvYhU4l4wbZs0pg34avr1XwKvaPrqRREyY74i0gr8GLewF7MfCzPkq6oDqlqvqo2q\n2oh7jeF2Vd3nTbhzJpPv8/O4Z+2ISD2uTXO8kEHmkEzyPQlsAxCRtbjF/VxBoywcbcDfprtmNgMD\nqvphXvbk9dXlDK9Ab8c9ezkG3J/Wvo/7Qw7ul+FZ4Cjwe2CF1zHnOd//A84AB9L/2ryOOZ/5Thr7\nO4q4WybD4yu4VlQH8C6ww+uY85xvM7AHt5PmAPBnXsc8h1yfAj4EErh/gd0N3AvcO+HYPpqei3fz\n+V02d6gaDAZDCVIMtozBYDAYssQUd4PBYChBTHE3GAyGEsQUd4PBYChBTHE3GAyGEsQUd4PBYChB\nTHE3GAyGEsQUd4PBYChB/h+KV/29wvYitQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5f1d545c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import itertools\n",
    "\n",
    "\n",
    "targetColorDict = {\n",
    "    0: 'red',\n",
    "    1: 'green',\n",
    "    2: 'blue'\n",
    "}\n",
    "\n",
    "# 绘制预测点\n",
    "for pt in itertools.product(np.arange(0,1,0.02), np.arange(0,1,0.02)):\n",
    "    t = KNN(data, np.array([pt[0], pt[1]]), K)\n",
    "    plt.scatter(np.array([pt[0]]),np.array([pt[1]]),s=25,marker='o', c=targetColorDict[t],alpha=0.2)\n",
    "\n",
    "# 绘制数据点\n",
    "for t,color in targetColorDict.items():\n",
    "    idxs = np.argwhere(target == t)\n",
    "    idxs.resize(len(idxs))\n",
    "    plt.scatter(data[idxs][:,0],data[idxs][:,1],s=25,marker='o', c=targetColorDict[t])\n",
    "\n",
    "\n",
    "plt.show()\n",
    "    \n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 引入四维数据\n",
    "\n",
    "之前我们只采用了两维数据，现在，我们可以引入四维数据：　花萼长度(sepal length)，花萼宽度(sepal width)，花瓣长度(petal length)，花瓣宽度(petal width)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = iris.data\n",
    "y = iris.target\n",
    "\n",
    "n_sample = len(y)\n",
    "\n",
    "# 初始化排序\n",
    "idxs = np.arange(n_sample)\n",
    "# 打乱排序\n",
    "np.random.shuffle(idxs)\n",
    "\n",
    "data = data[idxs]\n",
    "target = y[idxs]\n",
    "\n",
    "# 训练集\n",
    "X_train = data[:120]\n",
    "y_train = target[:120]\n",
    "# 测试集\n",
    "X_test = data[120:]\n",
    "y_test = target[120:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "准确率：　0.87\n"
     ]
    }
   ],
   "source": [
    "accuracy = evaluateModel(X_train,y_train, X_test, y_test,K=1)\n",
    "print(\"准确率：　%.2f\"%(accuracy))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "K=  1; Accuracy: 0.87\n",
      "K=  3; Accuracy: 0.87\n",
      "K=  5; Accuracy: 0.73\n",
      "K=  7; Accuracy: 0.70\n",
      "K=  9; Accuracy: 0.50\n",
      "K=  11; Accuracy: 0.40\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH9lJREFUeJzt3Xl4VdW9xvHvLxMhBBIDAYGEUUQhjAnIIOKEolZAcQAB\ntXVgEK3T9draa62t1dor1gEEtFaLIuCIVSyi5YICCglCGAMISMI8JUyBEFj3j0RAiyVAknWG9/M8\nPHJO9jn7ZT/mzc7a+6xlzjlERCR0RfgOICIiFUtFLyIS4lT0IiIhTkUvIhLiVPQiIiFORS8iEuJU\n9CIiIU5FLyIS4lT0IiIhLsrXjmvVquUaNWrka/ciIkEpKytrm3Mu+WRe463oGzVqRGZmpq/di4gE\nJTP77mRfo6EbEZEQp6IXEQlxKnoRkRCnohcRCXEqehGREKeiFxEJcSp6EZEQ5+0++s279jPi0xxf\nuw8o7RqewUXNa/uOISIhylvRb9l9gBemr/K1+4Dx/ZK991zSjPsubYaZ+Q0kIiHHW9G3qp9A5pNX\n+dp9wCgqPswj7y/i+c9XsnrrHv73+jbERkf6jiUiIcRb0UuJmKgInr6uNU1rx/Onfy4nb2chY29O\np3b1WN/RRCRE6GJsADAzhnRvyuiB6eRs2k2fF2exbOMu37FEJESo6API5S3P5O0hnTns4LqXZvP5\nss2+I4lICFDRB5i0+glMHt6VJsnx3P73TF75YjXu+yu2IiKnQEUfgOrUiGXS4M70bHkmf/h4Gb9+\nfzEHDx32HUtEgpSKPkBVjYlk5E3tueuiprw1dx23vDqXgn0HfccSkSCkog9gERHGf11+Ds9c34Z5\na3dwzahZrNm213csEQkyKvog0Dc9hTdv78TOfUVcM2oWX63e7juSiAQRFX2Q6Ng4iQ/u6kqt+CoM\n+uvXTJqX6zuSiAQJFX0QaVizGu8N60KnJjV56N1snpyyjMOHdUeOiPxnKvogUyM2mr/d2oFBnRoy\nZuZqBr+Rxd4Dxb5jiUgAU9EHoajICH7fJ43f9WrJ58s2c/3oOWwsKPQdS0QClIo+iN3SpRGv3tqB\ndTv20fvFWWTn5fuOJCIBSEUf5C5sXpt3h3YhJiqCG8bMYcqijb4jiUiAUdGHgOZnVueDu7rSsl4C\nw96cz8jpqzRtgogcoaIPEbXiq/Dm7efRp209/jw1hwcmLeRA8SHfsUQkAGg++hASGx3Jsze2pWly\nPM9MW0Huzn2MHphOzfgqvqOJiEc6ow8xZsbdlzTjxZvakZ1XQJ9Rs1i5ebfvWCLikYo+RP2sdT0m\nDu5MYdFhrh01mxkrtvqOJCKeqOhDWNvURCYP70pKUhy/eG0ef5+z1nckEfFARR/i6idW5Z0hnbmo\neTKPTl7Cbycvplhz24uEFRV9GKhWJYoxgzK4o1tjXp/zHb94PZNd+zW3vUi4UNGHicgI45GrWvDU\nta2YvWobfUfNJnfHPt+xRKQSlKnozaynmeWY2Soze/g4X29gZtPN7BszyzazK8s/qpSHfh0b8Pfb\nOrJl9wF6j5xF5todviOJSAU7YdGbWSQwErgCaAH0N7MWP9rsN8Ak51w7oB8wqryDSvnp0rQW7w/r\nQkLVaG56+Wve/ybPdyQRqUBlOaPvCKxyzq12zhUBE4DeP9rGATVK/54AbCi/iFIRmiTH8/6wLqQ3\nPIP7Ji7kmU9zNLe9SIgqS9HXB45dziiv9LljPQYMNLM8YApw9/HeyMzuNLNMM8vculX3dfuWGBfD\n67/oyI0Zqbzwr1Xc/dY3FBZp2gSRUFNeF2P7A68551KAK4FxZvZv7+2cG+ucy3DOZSQnJ5fTruV0\nxERF8FTfVjxy5blMWbyRfmPnsGXXft+xRKQclaXo1wOpxzxOKX3uWLcBkwCcc3OAWKBWeQSUimdm\n3HFBE8YOymDllj30HjmLJRsKfMcSkXJSlqKfBzQzs8ZmFkPJxdYPf7TNOuASADM7l5Ki19hMkOnR\nog5vD+kMwPWj5zBt6WbPiUSkPJyw6J1zxcBwYCqwjJK7a5aY2eNm1qt0sweAO8xsIfAWcKvThOhB\nqWW9BCbf1ZVmteO5c1wmY2d+q7ntRYKc+fomzsjIcJmZmV72LSdWWHSIB99eyMeLNnJjRiq/75NG\nTJQ+Xyfim5llOecyTuY1mo9ejqtqTCQv9G9Hk+RqvPCvVXy3Yy+jB6aTGBfjO5qInCSdoslPiogw\nHrisOc/e2Ib53+VzzajZrN66x3csETlJKno5oWvapTD+jvPYVXiQa0bNZva323xHEpGToKKXMslo\nlMQHd3WldvUq3PzXuUyYu853JBEpIxW9lFlqUhzvDutC17Nq8fB7i3ji46Uc0rQJIgFPRS8npUZs\nNH+9JYNbuzTi5S/WMHhcJnsPFPuOJSL/gYpeTlpUZASP9WrJ471bMj1nK9eNnsOG/ELfsUTkJ6jo\n5ZTd3LkRr97agbwd++g9chYLcvN9RxKR41DRy2npfnYy7w3rQmx0BDeOmcNH2ZqhWiTQqOjltDWr\nU50PhnWlVf0Eho//hhc+X6lpE0QCiIpeykXN+Cq8ecd5XNuuPs9MW8F9Exew/6DmthcJBJoCQcpN\nlahInrmhDU1rx/PnqTnk7ixkzKB0asVX8R1NJKzpjF7KlZlx10VnMWpAe5ZsKKDPyFnkbNrtO5ZI\nWFPRS4W4slVdJg3uTFHxYfq+NJvpOVt8RxIJWyp6qTCtUxKZPLwrDZLiuO21ebw2a40u0op4oKKX\nClU3oSpvD+nMJefW4bF/LOXRyUsoPnTYdyyRsKKilwpXrUoUYwamM7h7E8Z99R0/f20eBYUHfccS\nCRsqeqkUERHGr644l6f7tmbOt9vp+9Js1m3f5zuWSFhQ0UuluqFDKuNuO49tew7Qe+SXzF2zw3ck\nkZCnopdK17lpTd4f1pUz4mIY8MpXvJuV5zuSSEhT0YsXjWtV4/1hXenQKIkH3l7I0/9czmHNbS9S\nIVT04k1CXDSv/6Ij/Ts2YNT/fcuwN+dTWKRpE0TKm6ZAEK+iIyP44zVpNE2uxhNTltHxj5/RJiWR\nNqkJtElJpG1qIrVrxPqOKRLUVPTinZlxe7cmtKyXwD+yN7AwN5/RM1YfWaawbkJsafmX/ABoVT+B\n6rHRnlOLBA8VvQSMzk1r0rlpTQAKiw6xdGMBC3ILWJibz8K8fP65ZBMAZnBWcnxp8SfSNiWR5mdW\nJyZKI5Eix6Oil4BUNSaS9IZJpDdMOvLczr1FLMzLZ2FuAQvz8pm+fAvvlN6xExMVQct6NY4M97RJ\nTaRRzTjMzNc/QSRgmK+5RzIyMlxmZqaXfUtocM6Rt7OwtPxLfgAsWl9AYek8+AlVo2mdklBS/KVD\nP8nVNWWyBDczy3LOZZzMa3RGL0HLzEhNiiM1KY6fta4HQPGhw6zcsufIcM+C3AJG/d+3R8b76ydW\nPXKht01qIq3qJ1Ctir4NJLTpjF5C3r6iYpZs2MXC3HwWlP4AyN1RCECEQbPa1UvKv/TMv/mZ1YmO\n1Hi/BCad0YscR1xMFB0aJdGh0dHx/u17DpCdV8CC3Hyy8/KZtnQzkzJLxvurREWQVj/hB7d5NtR4\nvwQxndGLcHS8f0Fu/pFhn0XrC9h/sGRK5cS4aFqnJNI2JeHI3T5aIlF80Bm9yCk6drz/6jZHx/tX\nbN5z5GLvgtx8Xpy+le9naqifWLX0Dp+Ss/40jfdLgNIZvchJ2FdUzOL1peP9pT8A8nYeHe8/u071\nH3y4q3md6kRpvF/KUYWd0ZtZT+A5IBJ4xTn31I++/ixwUenDOKC2cy7xZIKIBIO4mCg6Nk6iY+Oj\n4/3b9hwgu/QOn4W5+UxduomJmbkAxEZHkFYv4Qcf7kpNqqrxfqlUJzyjN7NIYAXQA8gD5gH9nXNL\nf2L7u4F2zrlf/Kf31Rm9hCrnHOt27Csd7y/5cNfi9QUcKC4Z7z8jLvrIHT5tUxNpnZJATY33SxlV\n1Bl9R2CVc2516U4mAL2B4xY90B/47cmEEAklZkbDmtVoWLMavdvWB+DgocPkbNr9gw93zVixku/P\ns1KTqv7gU71p9RKoGhPp8V8hoaQsRV8fyD3mcR5w3vE2NLOGQGPgX6cfTSR0REeW3LKZVj+BAec1\nBGDPgWIWrz86l8836/L5KHsjAJERRv+Oqfy+d5qGeeS0lfctAv2Ad5xzx51U3MzuBO4EaNCgQTnv\nWiS4xFeJolOTmnRqUvPIc1t27yc7t4BPFm/ija/W0TQ5np93bewxpYSCshT9eiD1mMcppc8dTz/g\nrp96I+fcWGAslIzRlzGjSNioXT2WS1vEcvE5tdm1/yB/+HgZLerW4LxjfhiInKyy3Pc1D2hmZo3N\nLIaSMv/wxxuZ2TnAGcCc8o0oEn4iIoxnbmhDw6Q47ho/n00F+31HkiB2wqJ3zhUDw4GpwDJgknNu\niZk9bma9jtm0HzDB+boxXyTE1IiNZsygdAqLDjH0zSwOFGuZRTk1+sCUSID7ZNFGhr45n5vOa8Af\nr2nlO454diq3V+ojeyIB7opWdRnSvSnjv17HxHnrfMeRIKSiFwkCD152NuefVYv/mbyEhbn5vuNI\nkFHRiwSBqMgInu/fjuT4Kgx9I4ttew74jiRBREUvEiSSqsUwZlA62/cWcff4byg+dNh3JAkSKnqR\nIJJWP4EnrmnFnNXbeXpqju84EiQ0ebZIkLkuPYXsvHzGzlxNq/oJR+bPF/kpOqMXCUK/uaoFGQ3P\n4KF3ssnZtNt3HAlwKnqRIBQTFcGoAe2Jj41i8LhMCgoP+o4kAUxFLxKkateI5aUB7cnbWcj9Exdw\n+LA+lC7Hp6IXCWIZjZJ49OoWfL58C8//a6XvOBKgVPQiQW5Qp4b0bZ/CXz5byefLNvuOIwFIRS8S\n5MyMJ65Jo2W9Gtw7cQFrtu31HUkCjIpeJATERkcyemA6URHGkHFZ7D1Q7DuSBBAVvUiISE2K4/n+\n7Vi5ZTcPvZuNZgyX76noRUJIt2bJ/Nfl5/Bx9kZe+WKN7zgSIFT0IiFmSPcmXJF2Jk9+sozZq7b5\njiMBQEUvEmLMjD9f34YmyfEMf+sb1ucX+o4knqnoRUJQfJUoxgxKp6j4MEPfyGL/QS1DGM5U9CIh\nqmlyPCNuaEN2XgGPTl6si7NhTEUvEsIua3kmd198FpMy8xg/V8sQhisVvUiIu/fSs+l+djKPfbiE\n+et2+o4jHqjoRUJcZITxXL+21E2oytA3sti6W8sQhhsVvUgYSIyLYfTAdAoKD3LXm/M5qGUIw4qK\nXiRMtKhXg6eubc3ctTv445RlvuNIJdJSgiJhpE+7+izMy+dvs9bSJiWRPu3q+44klUBn9CJh5tdX\nnkvHxkk8/F42SzYU+I4jlUBFLxJmoiMjGHlTexKqRjPkjSzy9xX5jiQVTEUvEoaSq1fhpYHpbCrY\nzz0TFnBIyxCGNBW9SJhq3+AMftcrjZkrtvLstBW+40gFUtGLhLH+HVO5MSOVF6evYuqSTb7jSAVR\n0YuEMTPjd71b0iYlgQcmLeTbrXt8R5IKoKIXCXOx0ZG8NDCdKlERDB6XxR4tQxhyVPQiQr3Eqrxw\nUzvWbNvLg5MWaqbLEKOiFxEAujStxa+uOId/LtnE6BmrfceRclSmojeznmaWY2arzOzhn9jmBjNb\namZLzGx8+cYUkcpw2/mN+Vnruvx56nK+WLnVdxwpJycsejOLBEYCVwAtgP5m1uJH2zQDfgV0dc61\nBO6tgKwiUsHMjKeva02z2tW5+61vyN2xz3ckKQdlOaPvCKxyzq12zhUBE4DeP9rmDmCkc24ngHNu\nS/nGFJHKEhdTsgzhocOOIVqGMCSUpejrA7nHPM4rfe5YZwNnm9ksM/vKzHoe743M7E4zyzSzzK1b\n9WuhSKBqVKsaz/Vry5INu/j1e4t0cTbIldfF2CigGXAh0B942cwSf7yRc26scy7DOZeRnJxcTrsW\nkYpw8Tl1uPfSZrz3zXrGffWd7zhyGspS9OuB1GMep5Q+d6w84EPn3EHn3BpgBSXFLyJB7J6Lm3HJ\nObV5/B9Lmbd2h+84corKUvTzgGZm1tjMYoB+wIc/2uYDSs7mMbNalAzl6P4skSAXEWGMuLEtKWdU\nZdib89m8a7/vSHIKTlj0zrliYDgwFVgGTHLOLTGzx82sV+lmU4HtZrYUmA78l3Nue0WFFpHKk1A1\nmjGDMth7oJhhb86nqFjLEAYb83WRJSMjw2VmZnrZt4icvI+yNzB8/DcM6tSQ3/dJ8x0nbJlZlnMu\n42Reo6UERaRMfta6Htl5BYyduZrWKQlcn5F64hdJQNAUCCJSZg9d3pwuTWvyyAeLWbxeyxAGCxW9\niJRZVGQEL/RvR61qMQwel8WOvVqGMBio6EXkpNSMr8LoQels3XOAu9+aT/EhXZwNdCp6ETlprVMS\n+UPvNGat2s7/fqplCAOdil5ETskNHVIZcF4DRs/4limLNvqOI/+Bil5ETtmjV7egXYNEHnx7ISs3\n7/YdR36Cil5ETlmVqEheGpBOXEwUg8dlsWv/Qd+R5DhU9CJyWs5MiGXUgPas27GP+ycu5PBhzXQZ\naFT0InLaOjZO4pGrzuWzZZsZOX2V7zjyIyp6ESkXt3ZpRJ+29Rjx2Qqm52jtoUCioheRcmFmPHlt\na845swa/fOsbvtu+13ckKaWiF5FyUzUmkjED0zEzBo/LYl9Rse9IgopeRMpZg5pxPNevLTmbd/Mr\nLUMYEFT0IlLuLmxemwcva87kBRt4ddZa33HCnopeRCrE0O5NuaxFHf44ZRlfrdY6RD6p6EWkQkRE\nGM/c0IaGNeMYPn4+GwsKfUcKWyp6Eakw1WOjGTsoncKiQwx9Yz4Hig/5jhSWVPQiUqHOql2dZ25o\nw4LcfB77cKnvOGFJRS8iFa5nWl2GXdiUt+auY8Lcdb7jhB0VvYhUigcua063ZrV4dPISFuTm+44T\nVlT0IlIpIiOM5/u1o3aNKgx9I4ttew74jhQ2VPQiUmnOqBbD6IHp7NhbxPDxWoawsqjoRaRSpdVP\n4MlrW/HV6h089cly33HCgopeRCrdte1TuKVzQ175cg0fLtzgO07IU9GLiBePXNWCDo3O4L/fyWbZ\nxl2+44Q0Fb2IeBETFcHIm9pTPTaKIW9kUbBPyxBWFBW9iHhTu0YsLw1sz4b8Qu6d+I2WIawgKnoR\n8Sq9YRKPXt2S6Tlb+cvnK33HCUkqehHxbuB5DbguPYXnP1/JZ0s3+44TclT0IuKdmfGHPmmk1a/B\nfRMXsGabliEsTyp6EQkIsdGRjB6YTlSkceffM9l7QMsQlhcVvYgEjJQz4nihf3u+3bqHh97J1jKE\n5aRMRW9mPc0sx8xWmdnDx/n6rWa21cwWlP65vfyjikg4OL9ZLR7qeQ4fL9rIy1+s9h0nJESdaAMz\niwRGAj2APGCemX3onPvxxNITnXPDKyCjiISZwRc0ITsvn6c+WU7Legl0PauW70hBrSxn9B2BVc65\n1c65ImAC0LtiY4lIODMznr6uDU2T4xk+fj55O/f5jhTUylL09YHcYx7nlT73Y33NLNvM3jGz1HJJ\nJyJhK75KFGMGpVN8yHHNqNlMWbRRY/anqLwuxv4DaOScaw1MA14/3kZmdqeZZZpZ5tatW8tp1yIS\nqpokxzNhcCfq1KjCsDfnc8ffM9mQr0XGT1ZZin49cOwZekrpc0c457Y7575fReAVIP14b+ScG+uc\ny3DOZSQnJ59KXhEJMy3rJfDBsK785qpzmbVqOz1GzODVL9dwSNMllFlZin4e0MzMGptZDNAP+PDY\nDcys7jEPewHLyi+iiIS7qMgIbu/WhE/vu4CMRkk8/tFSrh01i6UbNOtlWZyw6J1zxcBwYColBT7J\nObfEzB43s16lm91jZkvMbCFwD3BrRQUWkfCVmhTHaz/vwPP927E+v5CrX/ySpz5ZTmHRId/RApr5\nuriRkZHhMjMzvexbRIJf/r4inpyynImZuaQmVeWJPq244OzQHxI2syznXMbJvEafjBWRoJQYF8Of\nrmvNW3d0Ijoigptfnct9ExewXYuO/xsVvYgEtc5NazLll9245+Kz+Ch7A5eOmME7WXm6FfMYKnoR\nCXqx0ZHcf1lzPr6nG02S43nw7YUMeOVr1moWTEBFLyIh5Ow61Xl7cGf+0CeNRXkFXP6XmYycvoqD\nhw77juaVil5EQkpEhDGwU0M+e6A7F59Tmz9PzeHqF75k/rqdvqN5o6IXkZBUp0YsLw1M5+WbM8jf\nd5C+L83mt5MXs3t/+C1CrqIXkZDWo0Udpt1/Abd0bsTfv/qOHiNm8umSTb5jVSoVvYiEvOqx0TzW\nqyXvDe1CYlw0d47LYsi4LDYV7PcdrVKo6EUkbLRrcAb/uPt8/rvnOUzP2UKPETMYN2cth0N83hwV\nvYiElejICIZe2JRP77uANqmJ/M/kJVw3ejYrNu/2Ha3CqOhFJCw1rFmNcbd1ZMQNbVizbS9XPf8F\nz3yaw/6DoTdvjopeRMKWmXFt+xQ+f+BCrm5Tjxf+tYornvuCOd9u9x2tXKnoRSTsJVWLYcQNbXnj\ntvM4dNjR/+WveOidheTvK/IdrVyo6EVESp3frBZT772AoRc25d3567nkmRlMXrA+6OfNUdGLiByj\nakwk/93zHD66+3xSkuL45YQF3PK3eeTuCN4FylX0IiLHcW7dGrw3tAuPXd2CrLU7uOzZmYyd+S3F\nQThvjopeROQnREYYt3ZtzLT7u9P1rFr8ccpyeo+cRXZevu9oJ0VFLyJyAvUSq/Lyzem8NKA9W3cf\noM/IWfz+o6XsPVDsO1qZqOhFRMrAzLiiVV2m3d+d/h0b8Ncv13DZszOZvnyL72gnpKIXETkJCVWj\neeKaVrwzpDNxMZH8/LV5DB8/ny27A3feHBW9iMgpyGiUxMf3dOOBHmfz6ZLNXPrMDCbMXReQ8+ao\n6EVETlFMVAR3X9KMT+7txrl1a/Dwe4vo9/JXrNqyx3e0H1DRi4icpqbJ8Uy4sxNP921NzqbdXPnc\nFzz32UoOFAfGvDkqehGRcmBm3NAhlc/u707PtDN59rMVXPX8l8xbu8N3NBW9iEh5Sq5ehef7t+Nv\nP+9AYdEhrh89h1+/v4iCQn9LGKroRUQqwEXNazPt/gu4o1tjJsxdx6UjZvBx9kYv8+ao6EVEKkhc\nTBSPXNWCD4efT50aVbhr/Hxufz2TDfmFlZpDRS8iUsHS6ifwwbCu/Oaqc5n97XZ6jJjBq1+u4VAl\n3YqpohcRqQRRkRHc3q0Jn953AR0aJ/H4R0u5dtQslmwoqPB9q+hFRCpRalIcf7u1Ay/0b8f6/EJ6\nvTiLJz9ZRmFRxd2KqaIXEalkZsbVberx2f3duT49hTEzVnPZX2Ywc8XWCtmfil5ExJPEuBie6tua\nCXd2Ijoigptfnct9Exewfc+Bct2Pil5ExLNOTWoy5ZfduOeSZnyUvYFLRszgnay8crsVs0xFb2Y9\nzSzHzFaZ2cP/Ybu+ZubMLKNc0omIhInY6Eju73E2U+7pxlnJ8Tz49kIGvPI1a7ftPe33PmHRm1kk\nMBK4AmgB9DezFsfZrjrwS+Dr004lIhKmmtWpzqTBnXnimjQW5RVw+V9mMnL6Kg6exhKGZTmj7wis\ncs6tds4VAROA3sfZ7vfAn4DAnZRZRCQIREQYA85ryGcPdOeSc2vz56k5/Oz5L5m/buepvV8ZtqkP\n5B7zOK/0uSPMrD2Q6pz7+JRSiIjIv6lTI5ZRA9J5+eYMdu0/SN+XZp/S+5z2xVgziwBGAA+UYds7\nzSzTzDK3bq2Y24hEREJNjxZ1mHZ/d27p3OiUXl+Wol8PpB7zOKX0ue9VB9KA/zOztUAn4MPjXZB1\nzo11zmU45zKSk5NPKbCISDiKrxLFY71antJry1L084BmZtbYzGKAfsCH33/ROVfgnKvlnGvknGsE\nfAX0cs5lnlIiEREpVycseudcMTAcmAosAyY555aY2eNm1quiA4qIyOmJKstGzrkpwJQfPffoT2x7\n4enHEhGR8qJPxoqIhDgVvYhIiFPRi4iEOBW9iEiIU9GLiIQ487EiOYCZ7QZyvOw88NQCtvkOESB0\nLI7SsThKx+Ko5s656ifzgjLdXllBcpxzms4YMLNMHYsSOhZH6VgcpWNxlJmd9IdRNXQjIhLiVPQi\nIiHOZ9GP9bjvQKNjcZSOxVE6FkfpWBx10sfC28VYERGpHBq6EREJcZVe9Gb2qpltMbPFlb3vQGNm\nsWY218wWmtkSM/ud70w+mdlaM1tkZgtO5c6CUGFmzUuPwfd/dpnZvb5z+WBmvzSzxaXfH2F3DI7X\nl2Z2fenxOHy8dT+Ox8cZ/WtATw/7DUQHgIudc22AtkBPM+vkOZNvFznn2obzrXTOuZzSY9AWSAf2\nAe97jlXpzCwNuIOSdavbAD8zs7P8pqp0r/HvfbkYuBaYWdY3qfSid87NBHZU9n4DkSuxp/RhdOkf\nXTSRY10CfOuc+853EA/OBb52zu0rXRdjBiUFFzaO15fOuWXOuZP6sKnG6D0zs0gzWwBsAaY55772\nnckjB3xqZllmdqfvMAGiH/CW7xCeLAa6mVlNM4sDruSHy5pKGfn8ZKwAzrlDQFszSwTeN7M051y4\nXr843zm33sxqA9PMbHnpGU1YKl26sxfwK99ZfHDOLTOzPwGfAnuBBcAhv6mCk87oA4RzLh+YThhf\nv3DOrS/97xZKxqQ7+k3k3RXAfOfcZt9BfHHO/dU5l+6cuwDYCazwnSkYqeg9MrPk0jN5zKwq0ANY\n7jeVH2ZWzcyqf/934DJKfnUPZ/0J32EbAEp/u8PMGlAyPj/eb6LgVOlDN2b2FnAhUMvM8oDfOuf+\nWtk5AkRd4HUzi6Tkh+4k59xHnjP5UoeSoSso+f9yvHPun34j+VP6w64HMNh3Fs/eNbOawEHgrtLf\nfMPG8fqSkouzLwDJwMdmtsA5d/l/fB99MlZEJLRp6EZEJMSp6EVEQpyKXkQkxKnoRURCnIpeRCTE\nqehFREKcil5EJMSp6EVEQtz/AygL3TuHJ1W8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5f1ad1d748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "k_list = np.arange(1, 13, 2)\n",
    "result = []\n",
    "for K in k_list:\n",
    "    accuracy = evaluateModel(X_train,y_train, X_test, y_test,K=K)\n",
    "    result.append(accuracy)\n",
    "    print('K=  %d; Accuracy: %.2f'%(K, accuracy))\n",
    "\n",
    "plt.xticks(k_list)\n",
    "plt.xlim(1, 11)\n",
    "plt.plot(k_list,result)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上图中的数据，　我们不难看出，针对鸢尾花数据集，K=1的时候，准确率最高．\n",
    "\n",
    "其实机器学习的准确率并不是唯一的评价模型的标准，只不过阿凯在教程里目前只提到了这一种，在之后的学习过程中，我们会逐渐提到更多的模型评价指标，例如roc等．"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary 总结\n",
    "\n",
    "**KNN优点**\n",
    "原理比较简单，无需参数估计，没有训练过程，针对多分类问题比较好．\n",
    "\n",
    "**KNN缺点**\n",
    "之前提到过，KNN算法属于惰性学习，每次输入样本的时候，需要匹配所有．因此如果样本量很多的话，计算量就会非常高.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
